JP2006141734A - Image processing apparatus, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus, method and program capable of quantitatively determining a sequence evaluation using a sequence feature quantity taking account of the correlation of mucosal microstructure components such as respective pits. <P>SOLUTION: This image processing apparatus is provided with a microstructure extracting means extracting the mucosal microstructure based on image signals capturing the image of a subject, a feature quantity calculation means calculating regional feature quantities in respective elements of the mucosal microstructure based on the mucosal microstructure, and a sequence evaluation means evaluating the sequence of the mucosal microstructure based on the regional feature quantities. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program for quantitatively evaluating the arrangement of mucous membrane microstructures based on a medical image obtained by an endoscope apparatus or the like.

  In recent years, an elongated insertion part can be inserted into a body cavity, and a solid-state imaging device or the like provided at the distal end of the insertion part can be used as an imaging means to observe a body cavity organ or the like on a monitor screen for examination or diagnosis Endoscopic devices are widely used. There is also an ultrasonic endoscope apparatus that can irradiate an organ in a body cavity with ultrasonic waves, observe the state of the organs in the body cavity from a monitor screen based on the reflection or transmission of the ultrasonic waves, and inspect or diagnose it. Widely used.

  The final diagnosis using these endoscopic devices largely depends on the subjectivity of the doctor, and it is desired to realize an endoscopic diagnosis support device that directly relates to objective and numerical diagnosis. It was.

  The endoscopic diagnosis support apparatus uses various feature amounts calculated from a region of interest (ROI) in an image, and an image to be diagnosed using threshold processing or a statistical / nonstatistic discriminator It provides support for objective and numerical diagnosis by presenting to the doctor whether it is classified as a finding.

  The feature amount is a numerical value reflecting various findings on the endoscopic image, and is obtained by applying an image processing method. In the diagnosis using endoscopic images, mucosal surfaces in endoscopic images such as dilation and meandering of blood vessels seen in fluoroscopic blood vessel images, size and inconsistency of the shape of gastric subdivision, and groove width of gastric subsection Structural findings are important elements for diagnosis of various diseases, and these can also be quantified as feature quantities by applying an image processing technique (see, for example, Patent Document 1).

  In recent years, the fineness of the mucosal surface structure and the pattern of the mucosal surface structure exhibited by the spatial frequency analysis of the Gabor feature calculated using a known Gabor filter with improvements applied to endoscopic images. There is an endoscopic image processing technique for digitizing the directionality of the image as a feature amount. As a feature value calculation method using this endoscopic image processing method, each finding is shown even when the mucosal surface structure of the lesion presents multiple different findings or when other different tumors are mixed in the lesion. Set the region of interest for each part, and know what findings or tumors are diagnosed in relation to the findings, and what proportions are mixed. A method has been proposed (see, for example, Patent Document 2).

On the other hand, with recent endoscope devices, high-quality images, solid-state image sensors (CCDs, etc.) with higher pixels, and magnification observation (zooming) while maintaining the same outer diameter and operability as ordinary endoscopes With the advent and spread of functional magnifying endoscopes, it has become possible to observe extremely fine capillaries on the mucosal surface and the pit pattern of the stomach and large intestine. Findings have been obtained, and it has been applied to diagnosis of lesion type and progress of cancer. Specifically, observation of the pit pattern of the large intestine (see, for example, Non-Patent Document 1 and Non-Patent Document 2) and IPCL (for example, see Non-Patent Document 3), which is an intramucosal microvascular finding in the esophagus, is known. It has been.
Japanese Patent No. 2918162 (FIG. 6) JP 2002-165757 A (FIG. 3) Shindo Kudo, “Depressed Early Colorectal Cancer”, Monthly Book Gastro, All Japan Hospital Press, May 15, 1993, Vol. 3, No. 5, p. 47-p. 54 Shindo Kudo, two others, “Diagnosis of Colorectal Diseases Using an Expanded Electronic Scope”, Stomach and Intestine, Medical School, February 26, 1994, Vol. 29, No. 3, p. 163-p. 165 Haruhiro Inoue and five others, “Enlarged Endoscopic Diagnosis of Esophageal Disease”, Gastrointestinal Endoscopy, Tokyo Medical, March 25, 2001, Vol. 13, No. 3, p. 301-p. 308

In the diagnosis using the large intestine pit pattern, the arrangement of pits is one of the important findings. For example, normal mucosa, regular type I, while inflammation and hyperplastic lesions shows the pit sequence of regular type II, tumor III _S type, III _L-type, IV type, V-type pit sequences Indicates. In particular, those showing regular type III _L are benign tumors, and those showing irregular type V are often cancers.

  However, Patent Document 1 does not disclose a method for evaluating the regularity of arrangement based on the arrangement of individual pits. Further, in Patent Document 2, statistical processing by frequency analysis is used as an endoscopic image processing method, and feature quantities reflecting elements other than regularity are calculated. There was a problem that it was difficult.

  Therefore, in the present invention, an image processing apparatus, an image processing method, and the like that enable quantitative array evaluation using array feature amounts based on an arrangement that takes into account the mutual relationship between mucosal microstructure components such as individual pits, An object of the present invention is to provide an image processing program.

  The image processing apparatus according to the present invention includes a fine structure extracting unit that extracts a mucosal microstructure based on an image signal obtained by imaging a subject, and a feature value that calculates a region feature value in each element of the mucosal microstructure based on the mucosal microstructure. Calculation means and arrangement evaluation means for evaluating the arrangement of the mucosal microstructure based on the region feature amount are provided.

  To realize an image processing apparatus, an image processing method, and an image processing program capable of quantitative array evaluation using an array feature amount based on an arrangement that takes into consideration the mutual relationship of mucosal microstructure components such as individual pits be able to.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
First, a network configuration with a related system of the image processing apparatus 1 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram schematically showing a network configuration of an image processing apparatus 1 and related systems according to the first embodiment of the present invention. As shown in FIG. 1, an image processing apparatus 1 according to the present embodiment is connected to a LAN 2 using TCP / IP as a communication protocol. On the other hand, an endoscope observation apparatus 3 that captures an image of a living body and outputs an image signal is also connected to the LAN 2 via an endoscope filing apparatus 4. The endoscope filing device 4 receives image signals from the endoscope observation device 3 to generate image data, and accumulates the generated image data. In other words, the image processing device 1 acquires the image data stored in the endoscope buffering device 4 via the LAN 2.

  Next, the overall configuration of the image processing apparatus 1 will be described. As shown in FIG. 2, the image processing apparatus 1 is configured around a general-purpose personal computer 11, and includes an operation device 12 including a keyboard and a mouse, a storage device 13 including a hard disk, and a display including a CRT. Device 14. FIG. 2 is a diagram schematically showing the overall configuration of the image processing apparatus 1. The operation device 12, the storage device 13, and the display device 14 are electrically connected to the personal computer 11, respectively.

  The personal computer 11 includes an external storage interface (hereinafter referred to as I / F) that reads and writes information between a CPU 21 that executes and controls various programs, a memory 22 that stores various processing programs and data, and a storage device 13. 23, a network card 24 that communicates information with an external device, an operation I / F 25 that receives an operation signal input from the operation device 12 and performs necessary data processing, and a display device 14 And a graphic board 26 for outputting a video signal to each other, and each is electrically connected to a bus 27. Accordingly, each unit of the personal computer 11 can transmit and receive information to and from each other via the bus 27.

  The network card 24 is electrically connected to the LAN 2 and transmits / receives information to / from the endoscope filing device 4 that is also connected to the LAN 2.

  The external storage I / F 23 reads the image processing program 28 stored in the storage device 13 and stores it in the memory 22. Note that the image processing program 28 is a program that executes image analysis processing, and includes a plurality of execution files, dynamic link library files, or setting files. When the image processing program 28 stored in the memory 22 is executed, the CPU 21 operates. The CPU 21 performs image analysis processing on the image data acquired from the endoscope filing device 4. Analysis data 29 acquired and generated by each process in the CPU 21 is stored in the memory 22. The analysis data 29 includes an original image 31 that is image data acquired from the endoscope filing device 4. Further, the analysis data 29 includes a binarized image 32, a pit centroid image 33, a Voronoi image 34, a region feature 35, and an array regularity feature 36. The images 32 to 34 and the feature amounts 35 and 36 will be described later in detail.

  The operation of the image processing apparatus 1 configured as described above will be described. In the present embodiment, a case will be described in which the arrangement of the mucosal microstructure is evaluated, for example, the arrangement regularity of a circular or short colon pit is evaluated.

  First, the image processing apparatus 1 acquires the original image 31 of the large intestine pit pattern imaged by the endoscope observation apparatus 3 via the image filing apparatus 4. The original image 31 is a color image in which each pixel is composed of 8-bit RGB. In general, when observing the colon pit pattern using the endoscope observation apparatus 3, in order to clearly observe the mucosal surface structure, a drug called a dye or staining agent such as indigo carmine or picotanine is used. . Since these drugs absorb most in the wavelength band of the R image, the subsequent image analysis processing is performed on the R image of the original image 31. When the original image 31 is acquired, the image processing program 28 of the image processing apparatus 1 is executed, and image analysis processing is executed according to the flowchart of FIG.

  FIG. 3 is a flowchart for explaining the procedure of image analysis processing by the image processing program 28. In addition to the processing steps, the flowchart of FIG. 3 also shows analysis data 29 input and output at each processing step. First, in step S1, a binarization process is performed on the original image 31 to generate a binarized image 32 in which the pixel value of the pixel corresponding to the pit portion is 1 and the pixel values of the other pixels are 0. . In the binarization processing in the present embodiment, the pixel value of each pixel in the original image 31 is set to 1 if the pixel value is equal to or greater than a preset threshold value, and if the pixel value is smaller than the threshold value. This is a process for setting the pixel value to 0. Note that, as pre-processing for binarization processing, inverse gamma correction, shading correction, and the like may be added to the original image 31. In addition, as the original image 31, a filtering image created by transmitting only a signal in a desired wavelength band by a known band pass filter (BPF) may be used, and binarization processing may be performed on the filtering image. .

  Next, in step S2, the binarized image 32 generated in step S1 is subjected to a labeling process to generate a labeling image 37. The labeling process in the present embodiment is performed as follows. First, the binarized image 32 is sequentially scanned from the upper left to the lower right, a pixel that is not labeled and has a pixel value of 1 is found, and an arbitrary symbol is assigned as a label. At this time, a label already assigned to another pixel is not used, and an unassigned label is assigned to the pixel. Next, the same label is assigned to all the pixels that are connected to the pixel to which the label is assigned and the pixel value is 1. The scanning and labeling described above are repeated until labels are assigned to all the pixels having a pixel value of 1. That is, by the labeling process, the same label is given to pixels belonging to the same connected portion, and a different label is assigned to each connected portion. Note that the processing in step S1 and step S2 corresponds to a fine structure extraction unit.

  Next, in step S3, a pit centroid calculation process is performed on the labeling image 37 generated in step S2 to generate a pit centroid image 33. The pit gravity center calculation process in the present embodiment is performed as follows. First, the average value of the position coordinates of the pixels assigned with the same label is calculated to obtain the center of gravity. Next, the pixel value of the pixel corresponding to the center of gravity is set as a label given to the pixel, and the pixel values of other pixels are set to 0, thereby obtaining the pit center of gravity image 33. That is, the pit centroid image 33 is an image showing the centroid of each pit included in the original image 31.

  Next, in step S4, the pit center of gravity image 33 generated in step S3 is Voronoi divided to create a Voronoi diagram, and a Voronoi image 34 is generated. The Voronoi diagram is a diagram created by Voronoi division, which is a well-known technique for dividing a certain plane into territory points called mother points arranged on the plane. Voronoi division is a method of drawing a vertical bisector between two adjacent generating points and forming a polygon by surrounding each generating point with such a line. Is divided into polygons, Takashi Takagi, “Mathematics of Shape”, Asakura Shoten, February 25, 1992, p. 43-p. 47 shows the details. Each side of the polygon formed around the generating point is called a Voronoi line. The region divided by Voronoi division is called the power range of the nearest point. Voronoi division in the present embodiment will be described with reference to FIG.

  4A and 4B are diagrams for explaining the Voronoi image 34 and various region feature amounts 35 calculated from the Voronoi image 34. FIG. 4A is a schematic diagram of the pit barycentric image 33, and FIG. 4 (a) is a Voronoi diagram obtained by dividing Voronoi, FIG. 4 (c) is a schematic diagram for explaining the power range of each pit, and FIG. 4 (d) is a schematic diagram for explaining the power range perimeter of each pit. 4 (e) is a schematic diagram illustrating the distance between the centers of gravity of adjacent pits in each pit. As shown in FIG. 4A, there are five pits in the pit centroid image 33 that is the basis of the Voronoi image 34, and the centroids of the pits are the mother point A41, the mother point B42, and the mother point. C43, generating point D44, and generating point E45 are shown.

  First, for all the pixels in the pit centroid image 33, the pit centroid image 33 is divided into five regions so that the pixels having the same nearest generating point are included in the same region, and are located on the boundary line of each region. The pixel value of the pixel is set to 1, and the pixel values of the other pixels are set to 0. With the above processing, a Voronoi diagram as shown in FIG. 4B is obtained. In the present embodiment, the Voronoi image 34 represents the power range of each element of the pit pattern. That is, as shown in FIG. 4C, the force range of the five pits that are the elements of the pit pattern is the pit force range having the center point A41 as the center of gravity and the pits having the center point B42 as the center of gravity. The force range of the pit is the region 47, the force range of the pit centered at the generating point C43 is the region 48, the force range of the pit centering on the generating point D44 is the region 49, and the force range of the pit centering on the generating point E45 is the region 50.

  Next, in step S5, a region feature 35 is calculated using the pit barycenter image 33 generated in step S3 and the Voronoi image 34 generated in step S4. In the present embodiment, as the region feature amount 35, the power range area, the power range perimeter, the distance between the centers of gravity of adjacent pits, and the number of adjacent regions are calculated. These region feature amounts 35 are calculated as follows.

  The power range area is the number of pixels in the power range, and is calculated for each pit. The power range area is calculated by counting the number of pixels belonging to the power range for each pit in the Voronoi image 34. The power range perimeter is the sum of the Voronoi line lengths surrounding the power range, and is calculated for each pit. For example, the force range length Al of the pit having the center A41 as the center of gravity is calculated as the sum of the lengths of Voronoi lines surrounding the region 46, as shown in FIG. The length of the Voronoi line that is the boundary with the region 47 (= a1), the length of the Voronoi line that is the boundary between the region 46 and the region 48 (= a2), and the Voronoi that is the boundary between the region 46 and the region 49 This is the sum of the length of the line (= a3) and the length of the Voronoi line (= a4) that is the boundary between the region 46 and the region 50.

  The distance between the centers of gravity of adjacent pits is the distance between the centers of gravity of two pits that touch the power range. For example, as shown in FIG. 4E, the pit having the center point A41 as the center of gravity and the region 46 as the power range is a region having the center point B42 as the center of gravity and the region 47 as the power range, and the center point C43 as the center of gravity. It is adjacent to four pits, a pit having a power range of 48, a pit having a center point D44 as a center of gravity and a region 49 as a power range, and a pit having a center point E45 as a center of gravity and a region 50 as a power range. Therefore, the distance between the centroids of the adjacent pits having the centroid at the generating point A41 is the distance between the generating point A41 and the generating point B42 (= d1) and the distance between the generating point A41 and the generating point C43 (= d2). The distance between the generating point A41 and the generating point D44 (= d3) and the distance between the generating point A41 and the generating point E45 (= d4) are four values.

  The number of adjacent regions is the number of power ranges that are adjacent to each other in a given power range with a Voronoi line surrounding the periphery as a boundary, and is calculated for each pit. For example, as shown in FIG. 4 (e), a pit having a center point A41 as a center of gravity and an area 46 as a power range is a pit having a base point B42 as a center of gravity and a region 47 as a power range, and a center point C43 as a center of gravity. Since it is adjacent to four pits, a pit having a power range of 48, a pit having a center point D44 as a center of gravity and a region 49 as a power range, and a pit having a center point E45 as a center of gravity and a region 50 as a power range, The number of adjacent areas is 4.

  When calculating these region feature values 35, the centroid of each pit is acquired from the pit centroid image 33 generated in step S3, and the power range of each pit is determined from the Voronoi image 34 generated in step S4. get. Note that the processing from step S3 to step S5 corresponds to a feature amount calculating means.

  Next, in step S6, the arrangement regularity feature quantity 36 as the arrangement feature quantity is calculated using the region feature quantity 35 calculated in step S5. In the present embodiment, as the arrangement regularity feature amount 36, the power range area, the power range perimeter, the distance between the centers of gravity of adjacent pits, and the coefficient of variation in each region feature amount 35 of the number of adjacent regions, Skewness and kurtosis are calculated.

  The variation coefficient CV is a scale for evaluating the degree of variation of the feature region amount 35, and takes a larger value as the variation is larger. That is, the smaller the pit arrangement, the smaller the value, and the more irregular, the larger the value. The variation coefficient CV is calculated by the equation (1).

式（１）において、ｘ（エックスバー）はｎ個の領域特徴量３５の平均値、ｓは標準偏差である。標準偏差ｓは式（２）にて算出される。 [Formula 1]

In Expression (1), x (X bar) is an average value of n region feature values 35, and s is a standard deviation. The standard deviation s is calculated by equation (2).

歪度β_１とは、領域特徴量３５の分布の非対称性を測る尺度であって、図５（ａ）に示すように、分布が右側に偏るときにはβ_１＜０、分布が左右対称のときにはβ_１＝０、分布が左側に偏るときにはβ_１＞０の値をとる。すなわち、pitの配列が規則的であるほど０に近い値をとる。図５（ａ）は、領域特徴量３５の分布と歪度との関係を説明する図である。歪度β_１は、式（３）にて算出される。 [Formula 2]

The skewness β ₁ is a scale for measuring the asymmetry of the distribution of the region feature 35, and as shown in FIG. 5A, when the distribution is biased to the right side, β ₁ <0, and when the distribution is symmetrical When β ₁ = 0 and the distribution is biased to the left, β ₁ > 0. That is, the closer to the pit arrangement, the closer to 0. FIG. 5A is a diagram for explaining the relationship between the distribution of the region feature 35 and the skewness. The skewness β ₁ is calculated by Expression (3).

式（３）において、ｎは領域特徴量３５の個数、ｘiは領域特徴量３５、ｘ（エックスバー）はｎ個の領域特徴量３５の平均値、ｓは標準偏差である。標準偏差ｓは、変動係数ＣＶと同様に式（２）にて算出される。 [Formula 3]

In Expression (3), n is the number of region feature values 35, xi is the region feature value 35, x (X bar) is an average value of n region feature values 35, and s is a standard deviation. The standard deviation s is calculated by the equation (2) similarly to the variation coefficient CV.

The kurtosis β ₂ is a scale for measuring the length (thickness) of the distribution of the region feature amount 35, and as shown in FIG. 5B, the length of the distribution that is wider than the normal distribution. Β ₂ > 3 when the distribution is long, β ₂ = 3 when the distribution is normal, and β ₂ <3 when the distribution is narrower than the normal distribution and short. That is, the closer to the pit arrangement, the closer to 0. FIG. 5B is a diagram for explaining the relationship between the distribution of the region feature amount 35 and the kurtosis. The kurtosis β ₂ is calculated by (Equation 4).

式（４）において、ｎは領域特徴量３５の個数、ｘ_iは領域特徴量３５、ｘ（エックスバー）はｎ個の領域特徴量３５の平均値、ｓは標準偏差である。標準偏差ｓは、変動係数ＣＶと同様に式（２）にて算出される。 [Formula 4]

In Expression (4), n is the number of area feature values 35, x _i is an area feature value 35, x (X bar) is an average value of n area feature values 35, and s is a standard deviation. The standard deviation s is calculated by the equation (2) similarly to the variation coefficient CV.

  Next, in step S7, the arrangement is determined using the arrangement regularity feature amount 36 calculated in step S6, specifically, the arrangement regularity is evaluated. The array regularity evaluation is performed according to the flowchart of FIG. Step S7 constitutes an array regularity determining means as an array determining means.

  FIG. 6 is a flowchart for explaining the sequence regularity evaluation procedure. First, in step S11, one or more arrangement regularity feature quantities 36 used for arrangement regularity evaluation are selected. Subsequently, in step S12, the number of arrangement regularity features 36 selected in step S11 is confirmed. If the number of selected arrangement regularity features 36 is one, the process proceeds to step S13, where the arrangement regularity features 36 selected in step S11 are compared with a preset threshold Th. The threshold value Th is set for each arrangement regularity feature amount 36 and is a determination condition as to whether or not the arrangement of pits is regular. For example, it is assumed that the variation coefficient of the power range area distribution is selected in step S11 and the corresponding threshold value Th is set to 0.3.

  As shown in FIGS. 7A and 7B, when the variation coefficient of the power range area distribution calculated in step S6 in FIG. 3 is less than 0.3, the process proceeds to step S14, and FIGS. As shown, the pit arrangement is determined to be regular, and the arrangement regularity evaluation ends. On the other hand, as shown in FIG. 7C, when the variation coefficient of the power range area distribution calculated in step S6 in FIG. 3 is 0.3 or more, the process proceeds to step S15, and as shown in FIG. , The pit arrangement is determined to be irregular, and the arrangement regularity evaluation is terminated.

  7A and 7B are diagrams for explaining the evaluation of arrangement regularity with respect to a pit pattern having a different coefficient of variation of the power range area distribution. FIGS. 7A and 7B show the coefficient of variation of the power range area distribution. Is a Voronoi image 34 having a value less than 0.3, FIG. 7D is a schematic diagram for explaining the Voronoi image 34 pit pattern shown in FIG. 7A, and FIG. 7E is shown in FIG. 7B. FIG. 7C is a schematic diagram for explaining the pit pattern of the Voronoi image 34. FIG. 7C shows the Voronoi image 34 in which the coefficient of variation of the power range area distribution is 0.3 or more. FIG. 7F shows the pit pattern shown in FIG. FIG. 6 is a schematic diagram illustrating a pit pattern of a Voronoi image 34.

  In step S12, when the number of arrangement regularity feature quantities 36 selected in step S11 is two or more, the process proceeds to step S16. In step S16, a discriminator is applied to the plurality of arrangement regularity feature quantities 36 selected in step S11. The discriminator uses discriminant analysis to classify data to which the discriminator is applied into one of a group called a predefined class. A feature vector for classifying the feature quantity into the class as an explanatory variable has been calculated using the feature quantity of the data already classified into a specific class, and the feature of the data whose class is unknown in this feature quantity vector By applying the quantity, the data is classified into any class. It is also possible to calculate an index (referred to as a distance from the class) indicating how similar the data is to each class. A class can be freely defined as long as it is a collection of data with similar attributes. In the present embodiment, the regularity of arrangement is defined as an attribute, and the regular group and irregular group of the pit pattern in the colonic mucosa are defined as classes. The arrangement regularity feature quantity 36 is used as an explanatory variable.

  The classifiers are roughly classified into a statistical classifier represented by Fisher's linear discriminant function and a non-statistical classifier represented by a neural network. It is appropriately selected and used in accordance with the distribution type (for example, multivariate normal distribution) of the feature quantity of the data to be classified and the performance of the classifier itself. In the present embodiment, Fisher's linear discriminant function is used as a discriminator, and a discriminant function g1 for a regular array class and a discriminant function g2 for an irregular array class are defined as discriminators. Accordingly, in step S16, the discriminant function g1 and the discriminant function g2 are applied to the plurality of arrangement regularity feature quantities 36 selected in step S11.

  Subsequently, in step S17, the application result of the discriminator in step S16 is confirmed. When the application result of the discriminant function g1 is larger than the application result of the discriminant function g2, the process proceeds to step S14, where it is determined that the pit arrangement is regular, and the arrangement regularity evaluation ends. On the other hand, when the application result of the discriminant function g1 is equal to or less than the application result of the discriminant function g2, the process proceeds to step S15, where it is determined that the pit arrangement is irregular, and the arrangement regularity evaluation ends. When the arrangement regularity evaluation shown in the flowchart of FIG. 6 is finished, all the image analysis processing shown in the flowchart of FIG. 3 are finished. Note that the processing of step S6 and step S7 corresponds to an array regularity evaluation means as an array evaluation means.

  Through the above processing, the image processing apparatus 1 acquires the original image 31 captured by the endoscope observation apparatus 3 via the image filing apparatus 4, and evaluates the arrangement regularity of the large intestine pit pattern in the original image 31. be able to.

  As described above, in the image processing apparatus 1 according to the present embodiment, the arrangement regularity feature amount of the pit can be quantitatively calculated in consideration of the relationship with the adjacent pit, and the arrangement regularity feature of the mucosal surface fine structure can be calculated. It is possible to evaluate the arrangement regularity based on the quantity.

  In the present embodiment, the evaluation of the arrangement regularity of pits in the large intestine mucosa has been described. For example, the arrangement regularity of pits in the esophageal fine blood vessels (IPCL) and the gastric mucosa is also evaluated. Can do.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The overall configuration of the image processing apparatus 1 is that the processing content of the image processing program 28 is different from that of the pit barycentric image 33 as analysis data acquired and generated by executing the image processing program 28 and stored in the memory 22. Since it is the same as that of the first embodiment except that the pit core line image 53 is included instead, only the characteristic operation will be described here, and the same components are denoted by the same reference numerals and the description will be omitted. Omitted.

  In the present embodiment, a description will be given of the evaluation of the arrangement of the mucosal microstructure, for example, the evaluation of the arrangement regularity such as a bowl-shaped colon pit. First, the image processing apparatus 1 acquires the original image 31 of the large intestine pit pattern captured by the endoscope observation apparatus 3 via the image filing apparatus 4 as in the first embodiment. When the original image 31 is acquired, the image processing program 28 of the image processing apparatus 1 is executed, and image analysis processing is executed according to the flowchart of FIG.

  FIG. 8 is a flowchart for explaining the procedure of image analysis processing by the image processing program 28. In addition to the processing steps, the flowchart of FIG. 8 also shows analysis data input / output at each processing step. First, in step S <b> 21, binarization processing is performed on the original image 31 to generate a binarized image 32. Next, in step S22, a labeling process is performed on the binarized image 32 generated in step S21 to generate a labeling image 37. The binarization process in step S21 and the labeling process in step S22 are respectively the binarization process in step S1 and the step S2 of the image analysis process in the first embodiment described with reference to FIG. This is the same process as the labeling process in FIG.

  Next, in step S23, the labeling image 37 generated in step S22 is subjected to a thinning process for calculating a pit core line, and converted into a thin line image having a pit pattern width of 1 pixel, thereby thinning the line. An image, that is, a pit core line image 53 is generated. The pit core line image 53 is an image representing a core line of a pit pattern, and the pixel value of the pixel corresponding to the core line is a label attached to the pixel, and the pixel values of the other pixels are zero. Here, the thinning process is a known process, and is performed as follows in the present embodiment. In the labeling image 37, as long as the connectivity of the pixels in each region formed by a group of pixels to which the same label is assigned, the contour pixels of the region, that is, the pixels having a pixel value of 0 The pixel values of adjacent pixels are successively set to 0. That is, the pixel value is set to 0 for each pixel layer sequentially from the outside in a region formed by a collection of pixels to which the same label is assigned, and the line width of the region is reduced until the line width of the region becomes 1. With the above processing, a pit core line image 53 having a line width of 1 is generated.

  Next, in step S24, the pit core line image 53 generated in step S23 is Voronoi divided to generate a Voronoi diagram. Voronoi division in the present embodiment is performed as follows. FIG. 9 is a diagram for explaining Voronoi division. FIG. 9A is a schematic diagram of a Voronoi image generated using the center of gravity of pit as a generating point, and FIGS. 9B and 9C are pit core lines. FIG. 9D is a schematic diagram for explaining the intermediate process of creating the Voronoi image 54 from the image 53, and FIG. 9D is a schematic diagram of the Voronoi image 54.

  First, in the pit core line image 53, a pixel assigned a label as a pixel value is used as a generating point, and the nearest generating point is calculated for all pixels. Next, the pit core line image 53 is Voronoi divided into the same number of regions as the number of generating points so that pixels having the same nearest generating point are included in the same region. In this embodiment, while sequentially scanning the pit core line image 53 from the upper left to the lower right, the pixels assigned with the labels as the pixel values are extracted every three pixels as the mother points, and the above-mentioned mother points are related to the above points. The Voronoi diagram as shown in FIG. 9B is obtained.

  Next, in step S25, the Voronoi diagram generated in step S24 is subjected to the same inter-label boundary line erasing process to generate a Voronoi image 54. As shown in FIG. 9C, the same-label boundary line erasure processing is performed by setting the pixel value located on the boundary line separating the regions of the same mother point in the Voronoi diagram generated in step S24 to 0. It is processing to do. By this process, a Voronoi image 54 representing the power range of each element of the pit pattern as shown in FIG. 9D is generated. In the case of a bowl-shaped large intestine pit, as described in the first embodiment, when Voronoi is divided using the center of gravity of the pit as a generating point, as shown in FIG. It may not be possible. However, in this embodiment, by performing Voronoi division using the core line of the pit as a generating point, the entire pit is always included in the pit's power range regardless of the shape of the pit, as shown in FIG. Such a Voronoi image 54 can be obtained.

  Next, in step S26, the region feature 35 is calculated using the Voronoi image 54 generated in step S25. The area feature quantity calculation process in step S26 is the same process as the area feature quantity calculation process in step S5 of the image analysis process in the first embodiment described with reference to FIG. The description about is omitted.

  Next, in step S27, the arrangement regularity feature amount 36 is calculated using the region feature amount 35 calculated in step S26. Since the arrangement regularity feature amount calculation processing in step S27 is the same processing as the arrangement regularity feature amount calculation processing in step S6 of the image analysis processing in the first embodiment described with reference to FIG. A description of specific processing is omitted.

  Next, in step S28, the arrangement regularity is evaluated using the arrangement regularity feature amount 36 calculated in step S27. The array regularity evaluation is performed according to the flowchart of FIG. 6 as in the first embodiment. When the arrangement regularity evaluation shown in the flowchart of FIG. 6 is completed, all the image analysis processes shown in the flowchart of FIG. 8 are completed.

  Through the above processing, the image processing apparatus 1 acquires the original image 31 captured by the endoscope observation apparatus 3 via the image filing apparatus 4, and evaluates the arrangement regularity of the large intestine pit pattern in the original image 31. be able to.

  As described above, in the image processing apparatus 1 according to the present embodiment, the arrangement regularity feature amount of the pit can be quantitatively calculated in consideration of the relationship with the adjacent pit, and the arrangement regularity feature of the mucosal surface fine structure can be calculated. It is possible to evaluate the arrangement regularity based on the quantity. In addition, by performing region division that reflects the shape of the pit, it is possible to evaluate a good arrangement regularity even for linear structural elements.

  In the present embodiment, the pit core line image 53 is generated in step S23 of FIG. 8, but a contour image of the pit may be generated, and this image may be applied to step S24 and subsequent steps. . The contour image is an image representing the contour of the pit, and in the labeling image 37 generated in step S22, for each region formed by a group of pixels to which the same label is assigned, The pixel value of the pixel having a pixel value of 0 is obtained by setting the pixel value of the pixel in the vicinity of eight as the label and setting the pixel values of the pixels other than the contour pixel to 0.

  In this embodiment, the evaluation of the arrangement regularity of pits in the large intestine mucosa has been described. However, as in the first embodiment, for example, the same applies to pits in the esophagus microvessel (IPCL) and gastric mucosa. The arrangement regularity can be evaluated.

  In the present embodiment, the arrangement regularity of the mucosal surface fine structure has been described as an example, but a series of image analysis techniques in the present embodiment can evaluate the distribution state of the mucosal surface fine structure, This doesn't stop there. For example, it is possible to identify the distribution of two types of patterns in which regularity is not recognized as long as the region feature amounts of the range of influence of each structural component are different.

  From the above embodiment, there is a feature in the points described in the following additional items.

(Supplementary Item 1) Inputting a medical image composed of one or more color signals obtained by imaging a biological mucosal surface image;
Extracting a mucosal microstructure from the medical image;
Calculating a region feature amount of the mucosal microstructure;
Evaluating the arrangement regularity of the mucosal structure from the region feature amount;
A medical image processing method comprising:

  (Additional Item 2) The medical image processing method according to Additional Item 1, wherein the step of extracting the mucosal microstructure includes the step of binarizing and extracting from at least one color signal of the medical image.

  (Additional Item 3) The step of extracting the mucosal microstructure further includes the step of identifying each element of the mucosal microstructure, and one or more pieces of structural information for each element identified in the step of identifying each element The medical image processing method according to supplementary note 1 or supplementary note 2, wherein the medical image processing method is extracted.

  (Supplementary Item 4) The step of calculating the region feature amount further includes a step of assigning a region to the structure information of each element, and calculating one or more region feature amounts from the assigned region. The medical image processing method according to appendix 4.

  (Additional Item 5) The medical image processing method according to Additional Item 4, wherein the step of allocating an area to the structural information of each element allocates an area according to a Voronoi diagram of the structural feature of each element.

  (Supplementary Item 6) The step of evaluating the arrangement regularity further includes a step of calculating one or more arrangement regularity features in an image using the one or more region feature amounts, and the arrangement regularity feature 6. The medical image processing method according to any one of appendices 1 to 5, wherein the arrangement regularity is evaluated based on the amount.

  (Additional Item 7) The medical image processing method according to Additional Item 3, wherein the step of identifying each element of the mucosal microstructure is identified by a labeling process of the mucous membrane microstructure.

  (Additional Item 8) The medical image processing method according to Additional Item 3 or Additional Item 7, wherein the structure information is a center of gravity of each element of the mucosal microstructure.

  (Additional Item 9) The medical image processing method according to Additional Item 3 or Additional Item 7, wherein the structure information is a thin line of each element of the mucosal microstructure.

  (Additional Item 10) The medical image processing method according to Additional Item 3 or Additional Item 7, wherein the structure information is a contour line of each element of the mucosal microstructure.

  (Additional Item 11) The medical image processing method according to Additional Item 5, wherein the region feature amount is an area of the allocated Voronoi region.

  (Additional Item 12) The medical image processing method according to Additional Item 5, wherein the region feature amount is a perimeter of the assigned Voronoi region.

  (Additional Item 13) The medical image processing method according to Additional Item 5, wherein the region feature amount is a distance between the centers of gravity of the assigned Voronoi regions.

  (Additional Item 14) The medical image processing method according to Additional Item 6, wherein the step of calculating the arrangement regularity feature amount calculates a variation coefficient indicating a variation of the region feature amount distribution. .

  (Additional Item 15) The medical image processing method according to Additional Item 6, wherein the step of calculating the arrangement regularity feature amount calculates a skewness representing the asymmetry of the region feature amount distribution.

  (Additional Item 16) The medical image processing method according to Additional Item 6, wherein the step of calculating the arrangement regularity feature amount calculates a kurtosis representing a length of the region feature amount distribution. .

  (Supplementary Item 17) The step of evaluating the arrangement regularity based on the arrangement regularity feature amount compares the one or more arrangement regularity feature amounts with a predetermined threshold value, whereby the arrangement of each element of the mucosal microstructure is determined. Item 18. The medical image processing method according to any one of Item 6, Item 14, Item 15, and Item 16, wherein it is determined whether the image is regular or irregular.

  (Supplementary Item 18) The step of evaluating the arrangement regularity based on the arrangement regularity feature amount applies the two or more arrangement regularity feature amounts to a discriminator, whereby the arrangement of each element of the mucosal microstructure is changed. Item 18. The medical image processing method according to any one of Item 6, Item 14, Item 15, and Item 16, wherein it is determined whether the image is regular or irregular.

  (Additional Item 19) The medical image processing method according to any one of Additional Item 1 to Additional Item 18, wherein the mucosal microstructure is a pit pattern in the medical image.

  (Additional Item 20) The medical image processing method according to any one of Additional Items 1 to 18, wherein the mucosal microstructure is a fine blood vessel in the medical image.

(Additional Item 21) An image input step of inputting a medical image composed of one or more color signals obtained by imaging a biological mucosal surface;
A mucosal microstructure extraction step for extracting the mucosal microstructure on the medical image;
A region dividing step of dividing the medical image into a plurality of regions based on the extracted mucous membrane microstructure;
A feature amount calculating step for calculating a feature amount from the divided area;
An identification step of identifying a mucosal surface state from the calculated feature amount;
A medical image processing method comprising:

  (Additional Item 22) The region dividing step further includes a reference information extracting step for extracting a reference point / or reference line of the mucosal microstructure, and an occupation range based on the reference point / or reference line from which the medical image is extracted. Item 22. The medical image processing method according to Item 21, which is divided.

  (Additional Item 23) The feature amount calculating step further includes a region feature amount calculating step of calculating a region feature amount from each divided region, and calculating a distribution feature amount based on the distribution of the calculated region feature amount. Item 23. The medical image processing method according to item 21 or 22,

  (Additional Item 24) The reference point / or the reference line of the mucosal microstructure is the center of gravity / or the fine line / or the contour line of the mucosal structure, according to the additional item 21, the additional item 22 or the additional item 23, Medical image processing method.

  (Additional Item 25) The medical image processing method according to Additional Item 23, wherein the region feature amount is an area / perimeter of the region.

  (Additional Item 26) The region feature amount calculating step further includes a step of specifying an adjacent structural component adjacent to the divided region, and the region feature amount is an adjacent structural component / or a distance between the centers of gravity of the divided regions / or an adjacent region. 26. The medical image processing method according to Additional Item 23 or Additional Item 25, wherein the medical image processing method is a number.

  (Additional Item 27) The additional feature item 23, wherein the distribution feature amount is a variation coefficient / or skewness / or kurtosis / or average value / or standard deviation / or variance of the region feature amount distribution. Medical image processing method.

  (Additional Item 28) The medical image processing method according to Additional Item 23, wherein the identifying step of identifying the mucosal surface state from the distribution feature amount compares one or more of the distribution feature amounts with a predetermined threshold value. .

  (Additional Item 29) The medical image processing method according to Additional Item 23, wherein the identifying step of identifying a mucosal surface state from the distribution feature amount applies two or more of the distribution feature amounts to a discriminator. .

  (Supplementary Item 30) The supplementary item 28 is characterized in that comparing the distribution feature amount with a predetermined threshold value determines whether the arrangement of each element of the mucosal microstructure is regular or irregular. The medical image processing method as described.

  (Additional Item 31) The application of the two or more distribution feature quantities to the discriminator is characterized by determining whether the arrangement of each element of the mucosal microstructure is regular or irregular. Item 29. The medical image processing method according to Item 29.

  (Additional Item 32) Determining whether the arrangement of each element of the mucosal microstructure is regular or irregular means determining whether the mucosal microstructure is included in a lesioned part. Item 32. The medical image processing method according to Item 31 or Item 31 characterized by the above.

  (Additional Item 33) The medical region according to any one of Additional Item 21 to Additional Item 32, wherein in the region dividing step, a region is assigned by a Voronoi diagram at a reference point / or reference line of the extracted mucosal microstructure. Image processing method.

  (Additional Item 34) The medical structure according to any one of Additional Item 21 to Additional Item 33, wherein the mucosal microstructure is a pit pattern / or microvessel in the surface image of the biological mucosa. Image processing method.

1 is a diagram schematically illustrating a network configuration between an image processing apparatus 1 and a related system according to a first embodiment of the present invention. 1 is a diagram schematically illustrating an overall configuration of an image processing apparatus 1. FIG. 4 is a flowchart for explaining a procedure of image analysis processing by an image processing program 28; 4A and 4B are diagrams for explaining the Voronoi image 34 and various region feature amounts 35 calculated from the Voronoi image 34. FIG. 4A is a schematic diagram of the pit centroid image 33, and FIG. 4B is FIG. FIG. 4C is a schematic diagram explaining the power range of each pit, FIG. 4D is a schematic diagram explaining the circumference of the power range of each pit, and FIG. 4E. These are the schematic diagrams explaining the distance between the centers of gravity of adjacent pits in each pit. FIG. 5A is a diagram for explaining the relationship between the distribution of the region feature 35 and the skewness, and FIG. 5B is a diagram for explaining the relationship between the distribution of the region feature 35 and the kurtosis. It is a flowchart explaining the procedure of arrangement | sequence regularity evaluation. 7A and 7B are diagrams for explaining the evaluation of arrangement regularity with respect to a pit pattern having a different coefficient of variation of the power range area distribution. FIGS. 7A and 7B show the coefficient of variation of the power range area distribution. 7 is a schematic diagram illustrating the pit pattern of the Voronoi image 34 shown in FIG. 7A, and FIG. 7E is a diagram illustrating FIG. 7B. FIG. 7C is a schematic diagram illustrating the pit pattern of the Voronoi image 34 shown, FIG. 7C is a Voronoi image 34 having a coefficient of variation of the power range area distribution of 0.3 or more, and FIG. 7F is FIG. It is the schematic explaining the pit pattern of the Voronoi image 34 shown in FIG. 5 is a flowchart for explaining a procedure of image analysis processing by an image processing program 51. FIG. 9A is a diagram for explaining Voronoi division. FIG. 9A is a schematic diagram of a Voronoi image generated using the center of gravity of pit as a generating point, and FIGS. 9B and 9C are Voronoi images from the pit core line image 53. FIG. 9D is a schematic diagram for explaining an intermediate process of creating the image 54, and FIG. 9D is a schematic diagram of the Voronoi image 54.

Explanation of symbols

1 image processing device,
11 Personal computer,
28 image processing program,
29 analysis data,
31 Original image,
32 Binary image,
33 pit centroid image,
34 Voronoi image,
35 area features,
36 sequence regularity features,
Agent Patent Attorney Susumu Ito

Claims (6)

Fine structure extraction means for extracting a mucous membrane fine structure based on an image signal obtained by imaging a subject;
Based on the mucosal microstructure, feature amount calculation means for calculating an area feature amount in each element of the mucosa microstructure,
An array evaluation means for evaluating the array of the mucosal microstructure based on the region feature amount;
An image processing apparatus comprising:   The fine structure extraction means includes: a component identification means for identifying a component of the mucosal fine structure; and a structure information extraction means for extracting one or more pieces of structure information for each component. The image processing apparatus according to claim 1.   The feature quantity calculating means includes area dividing means for assigning areas to each element of the micromucosal structure based on the structural information for each component of the micromucosal structure, and one or more area feature quantities for each area. The image processing apparatus according to claim 2, further comprising a region feature amount calculation unit for calculating.   The array evaluation unit includes an array feature amount calculating unit that calculates an array feature amount based on the region feature amount, and an array regularity determining unit that determines an array regularity of the mucosal microstructure based on the array feature amount. The image processing apparatus according to claim 1, wherein the image processing apparatus is characterized. A fine structure extraction step of extracting a mucous membrane fine structure based on an image signal obtained by imaging a subject;
Based on the mucosal microstructure, a feature amount calculation step for calculating an area feature amount in each element of the mucosa microstructure,
A sequence evaluation step of evaluating the sequence of the mucosal microstructure based on the region feature amount;
An image processing method comprising: On the computer,
A fine structure extraction procedure for extracting a mucous membrane fine structure based on an image signal obtained by imaging a subject;
Based on the mucosal microstructure, a feature amount calculation procedure for calculating an area feature amount in each element of the mucosa microstructure,
A sequence evaluation procedure for evaluating the sequence of the mucosal microstructure based on the region feature amount;
An image processing program for executing

Images