JP2007236956A - Endoscopic diagnosis support apparatus and endoscopic image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an endoscopic diagnosis support apparatus for utilizing all information that spatial frequency components have and making highly accurate objective/numerical diagnoses by applying a feature amount calculation method for which phase information is taken into consideration in turning the spatial frequency components in endoscopic images to a feature amount, and an endoscopic image processing method. <P>SOLUTION: A diagnosis support processing performance program 80 comprises an image input/management block 81, a database management block 82, a temporary ROI setting block 83, a feature amount calculation block 84, an ROI setting block 85, a discrimination and classification block 86, a report preparation block 87 and an image processing block 88. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an endoscopic diagnosis support apparatus and an endoscopic image processing method for automatically discriminating and classifying lesion types based on image data obtained by an endoscopic apparatus.

  2. Description of the Related Art In recent years, endoscope apparatuses that can insert an elongated insertion portion into a body cavity, use a solid-state imaging device or the like as an imaging means, observe a body cavity organ or the like on a monitor screen, and perform inspection or diagnosis have been widely used. Yes. There is also an ultrasonic endoscope apparatus that can irradiate the organ in the body cavity with ultrasound, observe the state of the organ in the body cavity on the monitor screen based on the reflection or transmission of the ultrasound, etc., and examine or diagnose it. Widely used.

  The final diagnosis using these endoscopic devices largely depends on the subjectivity of the doctor, and it is desired to realize an endoscopic diagnosis support device that directly relates to objective and numerical diagnosis. It was.

  The endoscopic diagnosis support apparatus uses various feature amounts calculated from a region of interest (ROI) in an image, and how an image to be diagnosed uses dark value processing or a statistical / nonstatistic discriminator. It provides support for objective and numerical diagnosis by presenting doctors with appropriate findings and lesions.

The feature amount is a numerical value reflecting various findings on the endoscopic image, and is obtained by applying an image processing method. For example, in the case where the findings relating to the color tone such as “redness of mucous membrane surface is red due to redness” are converted into feature quantities, R / (R + G + B) is obtained for each pixel using RGB data constituting the endoscopic image, and the average value thereof is obtained. Can be used as a feature amount (this feature amount is generally called chromaticity). In recent years, in the field of endoscope, a hemoglobin index obtained by 32 log ₂ (R / G) has been widely used as a color feature amount reflecting gastric mucosal blood flow.

  In addition, the findings on the mucosal surface structure in endoscopic images, such as dilation of blood vessels seen in fluoroscopic blood vessel images, meandering and size of the gastric subdivision, irregularity, groove width of the gastric subsection, etc. are also useful for diagnosis of various diseases. These are important elements, and these can be quantified as feature quantities by applying an image processing method. In such feature amount calculation, there is a method disclosed in Japanese Patent No. 2918162.

  Furthermore, by using each feature quantity obtained from a plurality of different findings as a feature vector, more complex and accurate diagnosis can be supported. In order to improve the accuracy of the endoscopic diagnosis support apparatus, it can be said that a highly accurate feature amount calculation method for quantifying important endoscopic image findings is very important.

  In recent years, the fineness of the mucosal surface structure and the mucosal surface structure are exhibited by a spatial frequency analysis technique in which the Gabor feature calculated using a known Gabor filter is improved for application to an endoscopic image. There is an endoscopic image processing method that digitizes pattern directionality as a feature amount.

  As an example of such an endoscopic diagnosis support apparatus and a feature amount calculation method, there is JP-A-10-14864.

  Recent endoscope devices have higher image quality, higher pixel count of solid-state image sensors (CCD, etc.), and also have an enlarged observation (zooming) function while maintaining the same outer diameter and operability as ordinary endoscopes. With the advent of the magnifying endoscope, extremely fine capillary images on the mucosal surface and the pit structure of the stomach and large intestine are clearly observed. It is possible to observe biological images at the same level as conventional tissue specimens under a stereomicroscope at the time of clinical examination under an endoscope, and establish new diagnostics using these microstructural observations. Has been.

As an example of this, Non-Patent Document 1 (Analysis of stereoscopic microscopic images of gastric elevated lesions and enlarged electronic endoscopic images from the viewpoint of histopathological findings, Toru Honda et al., Gastroenterological Endoscopy, May 1993, Volume 35 No. 5 pp.967-976), Non-patent document 2 (Depressed type early colorectal cancer, Shinhide Kudo, Mansleek Gastro, May 1993, Vol. 3, No. 5, pp. 47-53) and Non-patent document 3 ( Diagnosis of large intestine disease with an extended electronic scope, Nobuhide Kudo et al., Stomach and intestine, 1994, Vol. These indicate that lesions such as normal mucous membranes, benign tumors, and cancers can be distinguished from differences in the form and arrangement of pits observed on endoscopic images. For example, in Non-Patent Documents 2 and 3, pit patterns and arrangements found in large intestine lesions are classified into pit patterns composed of type I or type V and their subclasses, and are associated with various lesions. .
Japanese Patent No. 2918162 Japanese Patent Laid-Open No. 10-14864 Analysis of magnifying electron endoscopic image from gastroscopic lesions and histopathological findings, Toru Honda et al., Gastroenterological Endoscopy, May 1993, Vol. 35, No. 5, pp.967-976 Recessed type early colorectal cancer, Shinhide Kudo, Mansleek Gastro, May 1993, Vol. 3, No. 5, pp. 47-53 Diagnosis of large intestine disease with an expanded electronic scope, Shinhide Kudo et al., Stomach and intestine, 1994, Vol. 29, No. 3, pp.163-165

  However, in the realization of the objective and numerical diagnosis for these new endoscopic diagnostics, the diagnosis support apparatus disclosed in Japanese Patent Laid-Open No. 10-14864 has the following problems.

  In an endoscopic image, particularly in the case of close-up or magnified observation, the lesion is not limited to a case that always exhibits a uniform finding, but for example, even in a very small lesion having a size of several millimeters in the colon pit pattern, I is normal. Type, type IIIs of neoplastic lesions, and the type that can be said to be in between may be mixed. In addition, a lesion in which a part of a benign tumor has become cancerous, such as adenoma cancer, has been reported.

  Since information on the area that was not set as the area of interest cannot be obtained, for example, what type of pit pattern is present on the lesion mucosa, what percentage, or what kind of tumor is present in the lesion There is a problem that it is not possible to know whether or not there is a mixture.

  In addition, in calculating feature values from an endoscopic image, it is necessary to use more information in the endoscopic image in order to realize a more accurate diagnosis. On the other hand, in Japanese Patent Laid-Open No. 10-14864, the spatial frequency analysis method described above is applied, but these do not use all the information of the spatial frequency component. More specifically, information included in the spatial frequency component includes amplitude information and phase information constituting an endoscopic image, but there is a loss of information due to the fact that phase information is not used conventionally.

  Japanese Patent No. 2918162 discloses the calculation of the feature amount of the fine structure component on the mucosal surface, but there are the following problems when used in the endoscope diagnosis support apparatus.

  In clinical examinations using an endoscopic apparatus, various so-called pigments and dyes such as indigo carmine, methylene blue, and crystal violet are used to clearly observe the mucosal surface structure. For example, in observing the pit pattern of the large intestine, indigo carmine is used as a standard, and when more detailed observation is required, methylene blue or crystal violet is used.

  However, indigo carmine used as a so-called dye method and methylene blue and crystal violet used as a staining method have the following differences in properties.

  Indigo carmine does not have the property of being absorbed by the living body, and a clear observation image can be obtained by accumulating in a concave portion according to the uneven shape of the mucosal surface. On the other hand, methylene blue and crystal violet are, for example, absorbed by the nucleus in the cell, thereby making it possible to clarify the contrast with the part that has not been absorbed.

  Differences appearing on the image due to these properties will be described with reference to FIG. For example, the pit pattern of the large intestine is for observing the shape and arrangement of the glandular openings on the mucosal surface, but the glandular openings are minute holes that are open on the mucosal surface, so when indigo carmine is sprayed on the endoscope In the image, the glandular opening is observed as a blue-green region. When this is schematically shown, the region shown in black in FIG. On the other hand, in methylene blue and crystal violet, the glandular opening is naturally not stained because it is an empty space, and as shown in FIG. 23 (b), the state is opposite to that in FIG. 23 (a). In the mirror image, a region other than the glandular opening is observed as a blue region in methylene blue and a purple region in crystal violet.

  Therefore, for example, when extracting mucosal structures by threshold processing, the processing result is reversed due to the difference in each drug, and different feature values are calculated despite the same disease. There is a problem that normal images without spraying and images sprayed with various drugs cannot be used together.

  Further, in a normal endoscopic image without drug distribution, the structural component of the mucosal surface is contained most in the G image in the RGB image. The same applies to crystal violet because of its purple color. On the other hand, when a blue-type drug such as indigo carmine and methylene blue is used, the R image may be the most and contain structural components. Therefore, it is possible to improve the accuracy of diagnosis support by changing which of the RGB images to be processed in the feature quantity calculation according to the presence or absence of the medicine spraying and the type of confectionery used for the spraying. There is a problem that this point is not taken into consideration.

  The present invention has been made in view of the above circumstances, and by applying a feature amount calculation method considering phase information in the feature amount of the spatial frequency component in the endoscopic image, all the spatial frequency components have It is an object of the present invention to provide an endoscopic diagnosis support apparatus and an endoscopic image processing method capable of performing objective and numerical diagnosis with higher accuracy using the above information.

  An endoscope diagnosis support apparatus according to an aspect of the present invention includes a frequency component extraction unit that extracts a predetermined frequency component from at least one imaging signal among a plurality of imaging signals obtained by imaging a subject, and the frequency component extraction unit. Phase information detection means for detecting phase information based on the extracted predetermined frequency component; feature quantity calculation means for calculating the feature quantity of the subject based on the phase information detected by the phase information detection means; And feature amount calculation means for calculating the feature amount calculated by the feature amount calculation means.

  An endoscope image processing method according to another aspect of the present invention includes a step of inputting an endoscope image composed of a plurality of color signals, and filtering for extracting a spatial frequency component from the input endoscope image. A step of applying, a step of detecting phase information of the endoscopic image based on an application result of the filtering, and a step of calculating a feature amount based on the detected phase information.

  The endoscopic diagnosis support apparatus and the endoscopic image processing method according to the present invention apply a feature amount calculation method considering phase information in the feature amount of a spatial frequency component in an endoscopic image, thereby obtaining a spatial frequency component. It is possible to perform objective and numerical diagnosis with higher accuracy by using all the information possessed by.

  Embodiments of the present invention will be described below with reference to the drawings.

First embodiment:
In the first embodiment of the present invention, it is possible to set a region of interest (hereinafter referred to as ROI) for all lesions for which acquisition of diagnosis support information is desired. The present invention relates to an endoscopic diagnosis support apparatus capable of presenting at what rate such findings and diseases are present.

  1 to 12 relate to a first embodiment of the present invention, FIG. 1 is a block diagram showing a configuration of an endoscope diagnosis support apparatus, and FIG. 2 is a diagnosis support process execution in a conventional endoscope diagnosis support apparatus. FIG. 3 is a functional block diagram for explaining the configuration of the program, FIG. 3 is a flowchart for explaining the operation of the diagnosis support processing execution program in the conventional endoscope diagnosis support apparatus, and FIG. 4 is in the conventional endoscope diagnosis support apparatus. ROI setting window explanatory view for explaining ROI setting, FIG. 5 is a report display window explanatory view for explaining display of diagnosis support information in a conventional endoscopic diagnosis support apparatus, and FIG. FIG. 7 is a functional block diagram for explaining the configuration of a diagnosis support process execution program in the endoscope diagnosis support apparatus, and FIG. 7 is a diagnosis support process execution in the endoscope diagnosis support apparatus of the present embodiment 8 is a flowchart for explaining the operation of the program, FIG. 8 is a diagram for explaining the provisional ROI setting window for explaining provisional ROI setting in the endoscopic diagnosis support apparatus of the present embodiment, and FIG. 9 is a region division in the present embodiment. FIG. 10 and FIG. 11 are explanatory diagrams for explaining the region division processing in the present embodiment, and FIG. 12 is diagnosis support information in the endoscope diagnosis support apparatus of the present embodiment. It is report display window explanatory drawing for demonstrating the display of these.

  First, the configuration of the endoscope diagnosis support apparatus according to the first embodiment of the present invention will be described with reference to FIG. The configuration of the endoscopic diagnosis support apparatus according to the present embodiment is basically the same as that of a conventional endoscopic diagnosis support apparatus, but the blocks and operations that make up a diagnosis support program to be described later are made different. The purpose is achieved.

  As shown in FIG. 1, the diagnosis support apparatus 1 according to the present embodiment, for example, obtains an imaging signal from an electronic endoscope (not shown) and converts it into a video signal, and converts the video signal from the video processor 2 into a video signal. An observation monitor 3 to be projected; an input unit 4 that converts a video signal from the video processor 2 into image data and performs signal processing; image data that has undergone signal processing by the input unit 4; and a compressed image that is reversibly or irreversibly compressed. Server unit 5 for storing data, and image data or compressed image data stored in server unit 5 are retrieved and displayed, and a series of diagnostic support processes such as region-of-interest setting processing, feature amount calculation processing, discrimination classification processing, and the like And a conference unit 6 for performing the above.

  The input unit 4 includes an A / D converter 11 that converts an analog RGB video signal that is a video signal from the video processor 2 into image data that is a digital signal, and a memory that stores the image data, and management information from the video processor 2. An image processing unit 12 for generating an image file to which the image processor is added, a LAN controller 13 for sending the image file generated by the image processing unit 12 to the server unit 5 via a LAN (Local Area Network) cable 4a, and the video processor 2 And a controller 14 for controlling the image processing unit 12 and the LAN controller 13 as well as for communicating image data management information and the like.

  The video processor 2 includes a video signal output terminal, an analog RGB video signal output terminal, and a communication signal output terminal. The video signal output terminal is connected to the observation monitor 3, and the analog RGB video signal output terminal is an A / D converter. The communication signal output terminal is connected to the controller 14.

  The output terminal of the A / D converter 11 is connected to the data signal terminal of the image processing unit 12.

  The control signal end and the data signal end of the controller 14 are connected to the control signal end of the image processing unit 12 and the LAN controller 13 by the bus line 14a. The control of the image processing unit 12 and the LAN controller 13 by the controller 14 is performed by signals via the bus line 14a.

  For example, an observation image obtained by an electronic endoscope (not shown) converted into a video signal by the video processor 2 is displayed on the observation monitor 3 as an observation image. When the operator of the video processor 2 determines that the above-described observation image needs to be recorded, the above-described video signal is output to the A / D converter 11 as an analog RGB video signal, and this A / D converter 11 The analog RGB video signal is subjected to predetermined quantization processing, converted into a digital RGB video signal, and output to the image processing unit 12 as observation image data.

  The image processing unit 12 stores observation image data input from the A / D converter 11 under the control of the controller 14.

  The controller 14 performs various data processing such as reduction processing on the observation image data stored in the image processing unit 12, and further adds management information to form an image file, which is temporarily stored in the image processing unit 12 or a LAN. The data is output to the controller 13. Further, when the controller 14 temporarily stores the image file subjected to various data processing as described above in the image processing unit 12, the controller 14 outputs the image file from the image processing unit 12 to the LAN controller 13 at a predetermined timing. It has become.

  The server unit 5 includes a LAN controller 21 that receives an image file transmitted from the LAN controller 13 of the input unit 4, a memory 22 that temporarily stores the image file received by the LAN controller 21, and an image received by the LAN controller 21. A hard disk driver 24 that records a file on, for example, the hard disk 23 that is a large-capacity storage medium, and a compression device 25 that compresses the image file received by the LAN controller 21 reversibly or irreversibly and sends compressed image data to the hard disk driver 24. , A LAN controller 21, a memory 22, a hard disk driver 24, and a controller 26 for controlling the compression device 25. The hard disk driver 24 stores image files and compressed image data in hardware. And records the disc 23, LAN controller 21 via the LAN cable 5a has to send image files or compressed image data recorded in the hard disk 23 to the conference unit 6.

  The end of the LAN cable 4 a of the LAN controller 13 of the input unit 4 is connected to the LAN controller 21 of the server unit 5. This LAN cable 4a is a so-called 10baseT or 100baseT cable, and is capable of 10Mbit / sec bidirectional data communication within a length of 100m using a twisted pair cable. You can send and receive.

  The control signal terminal and the data signal terminal of the controller 26 of the server unit 5 are connected to the control signal terminal of the memory 22, the LAN controller 21, and the hard disk drive 24 by the bus line 26a.

  In the server unit 5, an image file from the LAN controller 13 of the input unit 4 is input by the LAN controller 21 via the LAN cable 4 a, and the controller 26 temporarily stores the screen file input via the LAN controller 21 in the memory 22. To remember. The hard disk drive 24 stores the image file in the hard disk 23, for example.

  The conference unit 6 receives an image file or compressed image data from the LAN controller 21 of the server unit 5 via the LAN cable 5a, and an image for storing the image file or compressed image data received by the LAN controller 31. The processing unit 32, the decompression device 33 that decompresses the compressed image data stored by the image processing unit 32, the image data that is a digital signal in the image file stored by the image processing unit 32, and the image data that is decompressed by the decompression device 22 Is converted to an analog RGB video signal, a D / A converter 34 that converts the analog RGB video signal, an observation monitor 35 that displays the analog RGB video signal converted by the D / A converter 34, and a controller 36 that controls the image processing unit 32. And.

  The data signal end of the image processing unit 32 is connected to the input end of the D / A converter 34, and the output end of the D / A converter 34 is connected to the observation monitor 35. A control signal end and a data signal end of the controller 36 are connected to control signal ends of the image processing unit 32 and the LAN controller 31 by a bus line 36a. A control signal end and a data signal end of the controller 36 are connected to a control signal end with a CPU 41 described later by a pass line 36b.

  In addition, the conference unit 6 inputs a CPU 41 for controlling the controller 36, a request for searching for an image file to the server unit 5, for example, and a keyboard 42 for inputting various information along with the image file. A keyboard interface (hereinafter referred to as a keyboard I / F) 43 that matches a signal of the keyboard 42 and a signal of the CPU 41, a search monitor 44 that displays information input by the keyboard 42, and a screen on the search monitor 44 A mouse 45 that gives an instruction to move the cursor coordinates to an arbitrary position, a mouse interface (hereinafter referred to as mouse I / F) 46 that matches the signal of the mouse 45 and the signal of the CPU 41, an execution program for the CPU 41, and Each image data on the menu screen of the search monitor 44 A hard disk 47 in which data is recorded, a hard disk interface (hereinafter referred to as a hard disk I / F) 48 for matching the signal of the hard disk 47 and the signal of the CPU 41, and a work memory 49 used as various processing work areas of the CPU 41; , An image processing unit 50 including a memory for storing a digital RGB video signal to be displayed on the search monitor 44, a printer 51 for printing information from the CPU 41, and a printer interface for matching the printer 51 and the CPU 41 (hereinafter referred to as printer I). / F) 52.

  Furthermore, the conference unit 6 includes a password storage unit 61 that stores a password among information input from the keyboard 42 by the CPU 41, a password monitoring unit 62 that determines the level of the password stored in the password storage unit 61, and a password monitoring unit. And a control restriction unit 63 that restricts the control of the CPU 41 based on the determination result 62.

  The control signal terminal and data signal terminal of the CPU 41 are connected to the hard disk I / F 48, the mouse I / F 46, the keyboard I / F 43, the work memory 49, the image processing unit 50 and the password monitoring unit 61 through the bus line 41a. Connected to the end. The CPU 41 controls the hard disk I / F 48, the mouse I / F 46, the keyboard I / F 43, the printer I / F 52, and the work memory 49 by the bus line 41a.

  The mouse I / F 46 detects a signal corresponding to the physical relative movement amount of the mouse 45 and outputs it to the work memory 49. The work memory 49 stores the above-mentioned movement amount.

  The keyboard I / F 43 outputs a signal of character information input from the keyboard 42 to the work memory 49, and the work memory 49 stores the character information described above.

  The hard disk I / F 48 reads out the program executed by the CPU 41 from the hard disk 47 and the image data for the search monitor 44 such as the menu screen, and outputs them to the work memory 49. The work memory 49 reads the image for the program and the search monitor 44 described above. Data etc. are memorized.

  The printer I / F 52 transmits information transmitted from the CPU 41 to the printer 51 and performs printing.

  As described above, the CPU 41 loads the program stored in the hard disk 47 into the work memory 44 when the power is turned on, and operates according to the program.

  As described above, the image file stored in the hard disk 23 of the server unit 5 is output to the hard disk driver 14 and temporarily stored in the memory 22 or directly to the LAN controller 21 under the control of the controller 26. An image file is input from the LAN controller 21 to the LAN controller 31 of the conference unit 6 via the LAN cable 5a.

  In the conference unit 6, the LAN controller 31 outputs an image file to the image processing unit 32 under the control of the controller 36. The image file stored in the image processing unit 32 from the hard disk 23 of the server unit 5 is separated into digital observation image data and management information under the control of the controller 36, and the digital observation image data is stored in the image processing unit 32. The management information is sent to the CPU 41 via the bus line 36b.

  The image processing unit 32 stores the above-described observation image data. As described above, the observation image data which is a digital signal stored in the image processing unit 32 is converted into an analog RGB video signal by the inverse quantization of the D / A converter 34 and is output to the observation monitor 35. ing. The observation monitor 35 displays the input analog RGB video signal as described above. If the observation image data is compressed image data, it is decompressed by the decompression device 33 and then output to the D / A converter 34.

  The CPU 41 performs arithmetic processing so that image data for the search monitor 44 such as a cursor by the mouse 45, character information by the keyboard 42, a menu screen from the hard disk 47, and management information are combined or displayed alone, and image processing is performed as image data. The information is stored in the unit 50.

  The observation image data of the image file recorded on the hard disk 23 is, for example, a color observation image is divided into horizontal 640 dots and vertical 480 dots, and RGB color signal levels corresponding to these dots are, for example, 8 bits. It is configured with a predetermined number of bytes quantized to be.

  Diagnosis support processing using the endoscope diagnosis support apparatus having the above-described configuration is executed by the CPU 41 using a diagnosis support processing execution program recorded in the hard disk 47. The diagnosis support process is operated by input using the keyboard 42 and the mouse 45 in a multi-window environment displayed on the search monitor 44.

  In the endoscopic diagnosis support apparatus, various information transmitted from the server unit 5 regarding the patient, examination, and image, the set position / shape information of the ROI, the accompanying finding information, and the feature amount calculated from the ROI A database for storing the value and the discrimination result obtained by applying the discrimination classification process is stored in the hard disk 47. This database is accessed by the diagnostic support process execution program.

  Next, an outline of a conventional diagnosis support process will be described with reference to FIG.

  As shown in FIG. 2, the diagnosis support processing execution program 70 includes an image input / management block 71, a database management block 72, an ROI setting block 73, a feature amount calculation block 74, a discrimination classification block 75, a report creation block 76, and an image processing. It consists of block 77. Hereinafter, the outline of each block constituting the diagnosis support processing execution program 70 will be described.

  The image input / management block 71 performs input of an endoscopic image used in diagnosis: support processing, tube embedding, search, and the like.

  The database management block 72 manages a database for storing and managing images used in diagnosis support processing, set ROIs, calculated feature amounts, and the like.

  The ROI setting block 73 sets an ROI for calculating a feature amount based on the mucosal surface structure on the endoscopic image.

  The feature quantity calculation block 74 calculates a feature quantity to be used for discrimination classification processing in the discrimination classification block 75 and statistical processing in the report creation block 76 by applying a feature quantity calculation method.

  The discrimination classification block 75 performs discrimination classification processing using the feature amount calculated in the feature amount calculation block 74.

  The report creation block 76 is based on a list display of each processing result obtained by the feature amount calculation block 74 and the discriminant analysis block 75, a statistical processing result display such as an average value of each feature amount for each lesion type, and a plot on a graph. Create a report.

  The image processing block 77 applies image processing such as noise suppression processing, inverse γ correction processing, and structural component enhancement processing to the endoscopic image used for the diagnosis support processing.

  Then, the processing result image by the image processing block 77 is recorded on the hard disk 23 in the server unit 5 in the same manner as the original image, and can be the target data for the diagnosis support processing.

  Next, the operation of the conventional endoscope diagnosis support apparatus will be described with reference to FIGS.

  In the following description, a series of processing is applied to G image data that well reflects the mucosal surface structure among R, G, and B image data constituting an endoscopic image.

  First, as shown in FIG. 3, in step S1, an endoscope image to be processed is input. The endoscopic image can be obtained by being input from the server unit 5 to the conference unit 6 via the LAN cable 5a or by being read by the database management block 72 being stored in the database in the hard disk 47.

  Next, a G image is obtained in step S2. Subsequently, noise suppression processing for reducing random noise on the image is applied in step S3.

  As the noise suppression processing, filtering using a general averaging filter, Gaussian filter, or median filter is applied to each pixel. At this time, the size of the filter (mask size) is determined by the degree of noise on the image. For example, when the noise is small, the size is about 3 × 3, and when the noise is large, a larger filter is used.

  Next, in step S4, inverse γ correction is applied to remove non-linear conversion for monitor display.

  Each process in steps S3 and S4 is executed in the image processing block 77 in FIG.

  Subsequently, in the ROI setting block 73, a series of processes shown in steps S5 to S9 is applied. First, a counter i for counting the number of ROIs set in step S5 is set to 1.

  Next, in step S6, the i-th ROI (i) is set on the G image data to be processed. For example, the ROI is set by operating the mouse 45 and the keyboard 42 on the ROI setting window 100 shown in FIG.

  In FIG. 4, the ROI is set using the mouse cursor 102 on the G image data displayed in the endoscope image display area 101. At this time, whether the ROI type is an arbitrary drawing shape or a fixed shape such as a rectangle is selected using the ROI type selection buttons 103a and 103b. Subsequently, in the ROI information input / display area 104, information associated with the ROI such as a lesion's visual form, findings, doctor's diagnosis name, comments, etc. is input or selected, and the ROI setting button 105 completes the ROI setting. The ROI setting window 100 includes an ROI deletion button 106 for deleting the set ROI, and an end button 107 for ending the ROI setting and closing the ROI setting window 100.

  Subsequently, the end of ROI setting is determined in step S7. If the next ROI is set without ending, the process proceeds to step S8. In the example using FIG. 4, this corresponds to the case where a new ROI is set on the endoscope image display area 101. If it is finished, the process proceeds to step S9. In the example using FIG. 4, this corresponds to the case where the end button 107 is pressed, and the ROI setting window 100 is closed. At this time, the database management block 72 saves the position / shape and the assigned information regarding the set ROI in the database in the hard disk 47.

  In step S8, the value of the counter i is incremented by 1, and the process returns to step S6.

  In step S9, the total number of set ROIs is N, and the process proceeds to step S10.

  In step S10, the feature amount is calculated from the set ROI (1) to ROI (N) by applying the feature amount calculation method in the feature amount calculation block 74. As the feature amount, for example, a statistic amount such as an average value of the feature amount using a Gabor filter disclosed in Japanese Patent Application Laid-Open No. 10-14864 is calculated. The calculated feature amount is stored in a database in the hard disk 47 after being associated with each of ROI (1) to ROI (N) by the database management block 72.

  In step S11, the discrimination classification for the set ROI (1) to ROI (N) is applied. Specifically, feature quantities obtained from lesion groups (for example, normal part, cancer, adenoma, etc.) that have already undergone ROI setting and feature quantity calculation (these correspond to classes in the multivariate analysis field) are represented by feature vectors ( These are called teacher data), and classification using a statistical or non-statistic classifier is applied in the discrimination classification block 75. The classifier is calculated by using, as teacher data, a feature vector based on a feature amount obtained from data in which a group (class) to which the class belongs is applied, and applying the data to any class It is classified into. It is also possible to calculate an index (referred to as a distance from the class) indicating how similar the data is to each class. A class can be freely defined as long as it is a set of data having similar attributes (findings correspond to this in the endoscopic diagnosis support apparatus in the present embodiment). In addition, for example, it is possible to classify the pit pattern in the large intestine mucosa (type I, type II,...) Or “grouped and irregular group”.

  Fisher's discriminant function is widely used as a statistical classifier, and neural network is widely used as a non-statistical classifier. The type of distribution of features from lesions to be classified (for example, multivariate) Whether the distribution is a normal distribution or other distribution) and the performance of the discriminator itself are appropriately selected and used. Here, Fisher's discriminant function is used.

  Finally, in step S12, the diagnosis support information including the discrimination classification result is displayed by the report creation block 76.

  A display example of the diagnosis support information is shown in FIG. The report display window 110 is opened, and the ROI number of the endoscopic image that is the target for displaying the diagnosis support information, the discrimination classification result by the discriminator, and the diagnosis name assumed from the discrimination classification result are displayed in the diagnosis support information display area 111. indicate. In this embodiment, the classification result (called a class) output from the discriminator is a gland opening (pit) on the mucosal surface, and the diagnosis name diagnosed from the type (pit pattern) is also displayed. . In the calculation of the discriminator, the diagnosis name itself can be a class.

  The report display window 110 includes a feature value display button 112, a graph display button 113, a print button 114, an image / ROI display button 115, and an end button 116, and feature values calculated by operating the mouse 45, respectively. Value display, average value display, feature value graph display, diagnosis support information printing, diagnosis support information display target image and ROI display, and diagnosis support information display end.

  Next, an outline of the endoscope diagnosis support apparatus in the present embodiment will be described with reference to FIG.

  As shown in FIG. 6, the diagnosis support processing execution program 80 in the present embodiment includes an image input / management block 81, a database management block 82, a temporary ROI setting block 83, a feature amount calculation block 84, an ROI setting block 85, It comprises a discrimination classification block 86, a report creation block 87, and an image processing block 88. The outline of each block constituting the diagnosis support process execution program 80 will be described below.

  The image input / management block 81 performs input of an endoscopic image used in the diagnosis support processing, tube embedding, search, and the like.

  The database management block 82 manages a database for storing and managing images used in diagnosis support processing, set ROIs, calculated feature amounts, and the like.

  The provisional ROI setting block 83 sets a region to be roughly processed (hereinafter referred to as provisional ROI) prior to setting an ROI to obtain detailed diagnosis support information on an endoscopic image.

  The feature amount calculation block 84 applies the ROI feature amount calculation method to the set temporary ROI, thereby setting the ROI in the ROI setting block 85 and the discrimination classification block 86 using the feature amount calculated for each ROI. The feature amount for use in the discriminant classification process in FIG. 5 and the statistical process in the report creation block 87 is calculated.

  The discrimination classification block 86 performs discrimination classification processing using the feature quantity calculated in the feature quantity calculation block 84 on the ROI set in the ROI setting block 85.

  The report creation block 87 is based on a list display of each processing result obtained by the feature amount calculation block 84 and the discriminant analysis block 86, a statistical processing result display such as an average value of each feature amount for each lesion type, and a plot on a graph. Create a report.

  The image processing block 88 applies image processing such as noise suppression processing, inverse γ correction processing, and structural component enhancement processing to the endoscopic image used for the diagnosis support processing.

  Then, the processing result image by the image processing block 88 is recorded on the hard disk 23 in the server unit 5 in the same manner as the original image, and can be the target data of the diagnosis support processing.

  Next, the operation of the endoscope diagnosis support apparatus according to the present embodiment will be described with reference to FIGS.

  In the following description, a series of processing is applied to G image data that well reflects the mucosal surface structure among R, G, and B image data constituting an endoscopic image.

  As shown in FIG. 7, first, in step S20, an endoscope image to be processed is input. The endoscopic image can be obtained by being input from the server unit 5 to the conference unit 6 via the LAN cable 5a or by being read by the database management block 82 being stored in a database in the hard disk 47.

  Next, a G image is obtained in step S21. Subsequently, in steps S22 and S23, the noise suppression process and the inverse γ correction process shown in steps S3 and S4 in FIG. 3 are applied. Application of each process in steps S22 and S23 is executed in the image processing block 88 in FIG.

  In the subsequent step S24, the temporary ROI setting block 83 determines whether or not to set a temporary ROI. If a temporary ROI is set, the process proceeds to step S25, and if not set, the process proceeds to step S26. The provisional ROI is for omitting the processing for the area other than the lesioned part, and the setting by the operator is not necessarily required in the present embodiment. When the temporary ROI is not set, the entire endoscopic image is set as the ROI setting target in the subsequent processing. In this case, the entire endoscope image is referred to as a temporary ROI in the following description.

  In step S25, a temporary ROI is set. The provisional ROI is for reducing the amount of calculation in the processing after step S26, and for avoiding the display of diagnosis support information for a region that is not a lesioned part. That is, the ROI for applying the original diagnosis support process is set in the set temporary ROI.

  The provisional ROI is set by, for example, operating the mouse 45 and the keyboard 42 on the provisional ROI setting window 200 shown in FIG.

  In FIG. 8, a temporary ROI is set using the mouse cursor 202 on the G image data displayed in the endoscopic image display area 201. Here, it is assumed that a figure indicated by a broken line in the endoscope image display area 201 is set as a temporary ROI. Whether the temporary ROI type is an arbitrary drawing shape or a fixed shape such as a rectangle is selected using the temporary ROI type selection buttons 203a and 203b. Subsequently, in the provisional ROI information input / display area 204, information associated with the provisional ROI such as the visual form of the lesion, the findings, the doctor's diagnosis name, and comments are input or selected, and the ROI setting is completed by the provisional ROI setting button 205. The temporary ROI setting window 200 includes a temporary ROI deletion button 206 for deleting the set temporary ROI, and an end button 207 for ending the temporary ROI setting and closing the temporary ROI setting window 200.

In subsequent step S26, by applying the feature amount calculation method in the feature amount calculation block 84, the feature amount is calculated for each surface element in the set temporary ROI. In the present embodiment, a Gabor feature value h (X, Y, θ _k , λ _m ) using a Gabor filter disclosed in Japanese Patent Laid-Open No. 10-14864 is used as a feature value. Here, X and Y are the coordinates of the pixel in the image, and θ _k and λ _m are parameters that define the direction and wavelength of the Gabor filter, respectively. In this embodiment, a Gabor filter having four directions and two frequencies is used, and 1 ≦ k ≦ 4 and 1 ≦ m ≦ 2. Eight feature values are calculated for each surface element by applying these Gabor filters. For the i-th pixel, Ci = {h1i, h2i, h3i,. . . , H8i} and vector notation (feature vector). i is an integer of 1 or more indicating the number of the pixel in the temporary ROI. Further, h1i to h8i indicate eight Gabor feature values calculated for the pixel i.

  Next, in step S27, the temporary ROI is divided into N (N is an integer of 1 or more) ROIs by region division processing using the feature values of each surface element calculated in step S26.

  The area division process in step S27 will be described with reference to FIG. The area division processing is to obtain a divided ROI by integrating the pixels in the temporary ROI using the fact that similar values are obtained from the pixels in the area showing the same findings. There are various methods for area segmentation processing using feature quantities. In this embodiment, a K-means algorithm, which is a typical technique for area segmentation by feature quantities (generally referred to as clustering), is used. To do.

In the present embodiment, the feature amount of each pixel calculated in step S26.
Ci = {h1i, h2i, h3i,. . . , H8i}
Are used in a series of processes after step S131 shown in FIG.

  In step S131, an initial area is set for the temporary ROI. FIG. 10 shows an example of setting the initial area. Here, the rectangle is mechanically set to a size of a maximum size L × L. However, the shape is not necessarily rectangular, and an area including the boundary line of the temporary ROI (solid line part in FIG. 10) is used in the vicinity of the temporary ROI boundary part. The value of L is determined based on the processing target. For example, if the observation image is a large intestine pit pattern, L = 5.

In the following step S132, the average value Cmeanj = {h1meanj, h2meanj,..., H8meanj} of the feature amount for each region is calculated. Here, j is an integer of 1 or more indicating the number of the set area.

  In step S133, Ci of each pixel is compared with Cmean of each region by the distance shown in the following equation.

ここで，α1ないしα8は各特徴量に対する重み付け係数を示す。ここでは均等にα1＝α2＝．．．＝α8＝１であるものとする。各領域ごとに得られたＤijに基づき、最小のＤijを与える領域に画業ｉを統合することで、領域の再構成を行う。 [Equation 1]

Here, α1 to α8 indicate weighting coefficients for each feature amount. Here, α1 = α2 =. . . = Α8 = 1. Based on the Dij obtained for each area, the area i is integrated into the area that gives the minimum Dij, thereby reconstructing the area.

  In step S134, as in step S132, an average value of feature values for each region after reconstruction is calculated.

  In step S135, convergence is confirmed by a change in the average value of the feature values for each reconstructed region. If the change amount is greater than or equal to the threshold value, the process returns to step S133 to reconstruct the area again. If it is less than the threshold value, it is considered that the reconstruction of the area has converged, and the process proceeds to step S136 to complete the area dividing process.

Here, the threshold for performing the convergence determination is a value that is experimentally determined based on the target endoscopic image and the feature amount, and is, for example, 10 .0 relative to the absolute value of the difference of Cmeanj before and after reconstruction. Set to 0.

  FIG. 11 shows an example of ROI setting by applying the area division processing. A temporary ROI 210 indicated by a broken line in FIG. 11 is divided into four ROIs. Each ROI is an area that is determined to have a different tendency from the adjacent ROI in the calculated feature amount. For the set N (N is an integer equal to or greater than 1) ROI (1) to ROI (N), the feature value for each ROI is calculated by the feature value calculation block 84 in step S28 of FIG.

  The difference between the feature amounts calculated in steps S26 and S28 will be described. The feature values in step S26 are calculated for each surface element, and the pixels in the temporary ROI are integrated using the fact that similar values are obtained from the pixels in the region showing the same findings, and ROI (i) (1 ≦ It is used to obtain i ≦ N). On the other hand, the feature value in step S28 is calculated for each obtained ROI (i), and is, for example, the average value or variance of the feature values of each surface element in each ROI, which is used in the subsequent classification processing It is. The feature amount calculation method applied in each step may be the same or different, and is selected according to the type of lesion to be diagnosed.

  In the present embodiment, the mean values μ (1) to μ (N) and standard deviations σ (1) to σ (N) of the feature amounts using the Gabor filter of the pixels included in each ROI calculated in step S26. Are calculated as feature values of ROI (1) to ROI (N).

  Subsequently, a series of processing shown in steps S29 to S32 is applied in the discrimination classification block 86 using the feature amount calculated for each ROI. First, in step S29, the value of the counter i is set to 1.

  In step S30, a discriminant classification result is obtained by applying a discriminator to ROI (i) in the same manner as shown in step S11 in FIG.

  In step S31, it is determined whether or not the processing has been completed for all N ROIs. If not completed, the value of i is incremented by 1 in step S32, and the process returns to step S30. If completed, the process proceeds to step S33.

  In step S33, the diagnosis support information including the discrimination classification result is displayed by the report creation block 87.

  A display example of the diagnosis support information is shown in FIG. The report display window 220 is opened and the ROI number of the endoscopic image for which diagnosis support information is displayed, the discrimination classification result by the discriminator, the diagnosis name assumed from the discrimination classification result, and the area of each ROI in the temporary ROI The ratio is displayed in the diagnosis support information display area 221.

  The report display window 220 includes a feature value display button 222, a graph display button 223, a print button 224, an image / ROI display button 225, and an end button 226, and feature values calculated by operating the mouse 45, respectively. Value display, average value display, feature value graph display, diagnosis support information printing, diagnosis support information display target image and ROI display, and diagnosis support information display end.

  As described above, according to the endoscopic diagnosis support apparatus of the present embodiment, it is possible to set an ROI for all lesions for which it is desired to obtain diagnosis support information, and each ROI. It is possible to present what kind of findings and diseases are present at each rate.

  In the present embodiment, calculation of a feature amount from a G image has been described, but an R or B image can also be used. It is also possible to use a luminance image obtained by conversion using each color signal. Furthermore, it is also possible to calculate a feature amount by combining a plurality of images such as R / (R + G + B) or R / G.

  Further, the feature quantity calculation method used for the region division processing for setting the ROI for the temporary ROI may be different from the feature quantity calculation method used for the identification classification for the ROI after setting. In that case, for example, R / (R + G + B) is used for the region division processing, and a feature amount based on the Gabor filter is used for the identification classification processing.

  In addition, information such as the average value of the entire temporary ROI of the feature amount, the standard deviation, and the identification classification result may be displayed as the diagnosis support information.

Second embodiment:
In the second embodiment of the present invention, all ROIs are set for lesions for which diagnosis support information is desired to be acquired, and what findings and diseases exist for each ROI and in what ratio. The present invention relates to an endoscopic diagnosis support apparatus capable of presenting more comprehensive diagnosis support information for lesions based on diagnosis support information for each ROI.

  The configuration of the endoscope diagnosis support apparatus and the configuration block of the diagnosis support processing execution program in the present embodiment are the same as those in the endoscope diagnosis support apparatus described in the first embodiment. Therefore, different points will be described below.

  FIGS. 13 to 16 relate to the second embodiment of the present invention, FIG. 13 is a block diagram for explaining the configuration of the diagnosis support processing execution program in this embodiment, and FIG. 14 shows the operation of the diagnosis support execution program. 15 is a flowchart for explaining the derivation of diagnosis support information. FIG. 16 is a report display window for explaining the display of diagnosis support information in the endoscope diagnosis support apparatus according to this embodiment. It is explanatory drawing.

  As illustrated in FIG. 13, the diagnosis support process execution program 230 includes an image input / management block 231, a database management block 232, a temporary ROI setting block 233, a feature amount calculation block 234, an ROI setting block 235, a discrimination classification block 236, It comprises a comprehensive diagnosis block 237, a report creation block 238, and an image processing block 239. Among these, the blocks other than the comprehensive diagnosis block 237 and the report creation block 238 perform the same processing as the blocks in the diagnosis support processing execution program 80 described in the first embodiment.

  The comprehensive diagnosis block 237 acquires comprehensive diagnosis support information for the entire lesioned part where the temporary ROI is set based on the discrimination classification result of each ROI obtained by the discrimination classification block 236.

  In addition, the report creation block 238 in the present embodiment displays comprehensive diagnosis support information obtained by the comprehensive diagnosis block 237 in addition to the display of diagnosis support information shown in the first embodiment.

  Next, the operation of the endoscope diagnosis support apparatus in the present embodiment will be described with reference to FIG. In the present embodiment, steps S20 to S32 of the operation described with reference to FIG. 7 in the first embodiment are the same.

  After applying the discriminant classification process to all ROIs in step S31, the process proceeds to acquisition of comprehensive diagnosis information in step S41. Here, the comprehensive diagnosis information is diagnosis support information for the entire provisional ROI obtained as follows.

  For example, in the diagnosis based on the form of the large intestine pit pattern, what type of pit is present and what ratio is an important diagnostic index. Therefore, the determination as shown in FIG. 15 is performed based on the discriminant classification information obtained from each ROI.

  In FIG. 15, it is determined in order from the disease having the higher malignancy whether each pit pattern exists or its ratio from step S51. In addition, the numerical value shown as the probability in FIG. 15 is a numerical value indicating how often the determined disease appears with respect to the discrimination classification result. For example, when steps S51, S52 and S53 are passed, it indicates that "the probability that the V-type pit pattern is an early stage cancer in which a lesion present at a rate of 20% or more has infiltrated under the mucous membrane is 60% or more" It is.

  In step S42 of FIG. 14, the report creation block 87 displays diagnosis support information including comprehensive diagnosis information. A display example of diagnosis support information is shown in FIG. In the report display window 240, in addition to the items described with reference to FIG. 12 in the first embodiment, comprehensive diagnosis information is displayed in the comprehensive diagnosis information display area 241.

  As described above, according to the endoscopic diagnosis support apparatus in the present embodiment, an ROI is set for all lesions for which acquisition of diagnosis support information is desired, and what findings and diseases are set for each ROI. In addition, it is possible to present the comprehensive diagnosis support information for the lesion based on the diagnosis support information for each ROI.

Third embodiment:
The third embodiment of the present invention is another embodiment of the endoscopic diagnosis support apparatus shown as the second embodiment, and the ROI is set for all lesions for which acquisition of diagnosis support information is desired. In addition, it is possible to present what kind of findings and proportions exist for each ROI, and more comprehensive diagnosis support information for lesions based on diagnosis support information for each ROI The present invention relates to an endoscopic diagnosis support apparatus capable of presenting.

  The configuration of the endoscope diagnosis support apparatus and the configuration block of the diagnosis support process execution program in the present embodiment are the same as those in the endoscope diagnosis support apparatus described in the second embodiment. Therefore, different points will be described below.

  FIG. 17 is an explanatory diagram of a report display window for explaining display of diagnosis support information in the endoscope diagnosis support apparatus according to the third embodiment of the present invention.

  In the second embodiment, the overall diagnosis information of the entire lesion is obtained by referring to the determination criteria illustrated in FIG. 15, but in the present embodiment, the following processing is performed in step S41 of FIG. To obtain comprehensive diagnostic information.

By the series of processing shown in steps S20 to S31 in FIG. 7, the discrimination classification result for each of the N ROIs and the area ratio occupied in the temporary ROI are calculated. In the present embodiment, the discrimination classification process is applied again using the area ratio with respect to the obtained discrimination classification result as a feature amount. For example, assuming seven types of type I, II, IIIs, IIIL, IV, Va, and VN in the classification of colon pit patterns, a 7-dimensional feature vector is determined by the area ratio value of each pit pattern. Can be configured. If the area ratio occupied by each pit pattern is 0.2, 0.0, 0.3, 0.1, 0.0, 0.2, 0.2, the feature vector X is
X = {0.2, 0.0, 0.3, 0.1, 0.0, 0.2, 0.2}
It becomes. Therefore, by applying to a discriminator created using teacher data with the diagnosis ratio of a disease such as cancer and glandularity as a class and the area ratio of each pit pattern occupying the lesion as a feature vector, It becomes possible to indicate which disease is classified.

  As described above, according to the endoscopic diagnosis support apparatus in the present embodiment, an ROI is set for all lesions for which acquisition of diagnosis support information is desired, and what findings and diseases are set for each ROI. In addition, it is possible to present the comprehensive diagnosis support information for the lesion based on the diagnosis support information for each ROI.

Fourth embodiment:
The fourth embodiment of the present invention does not display information that is not important for diagnosis in the display of diagnosis support information for the endoscope diagnosis support apparatus shown as the first, second, and third embodiments. The present invention relates to an endoscopic diagnosis support apparatus that can present only important information.

  The configuration of the endoscope diagnosis support apparatus and the configuration block of the diagnosis support process execution program in the present embodiment are the same as those in the endoscope diagnosis support apparatus described in the second embodiment. Therefore, different points will be described below.

  For example, in a diagnosis based on the morphology of the large intestine pit pattern, the type I pit pattern is a finding of normal mucosa and may not be important information as compared with other pit patterns. Therefore, in the display of the diagnosis support information in step S42 shown in FIG. 14, the display of the diagnosis support information for the ROI in which the discrimination classification processing result is determined to be the I type pit pattern as shown in FIG. 17 is omitted.

  At this time, a discrimination classification result for omitting information display is registered in a database stored in the hard disk 47 or a dedicated setting file, and is referred to at the time of report creation by the report creation block 238.

  As described above, according to the endoscopic diagnosis support apparatus of the present embodiment, important diagnosis support information is selectively presented even when, for example, a very large number of ROIs are set for the temporary ROI. It becomes possible.

Fifth embodiment:
The fifth embodiment of the present invention relates to the endoscopic diagnosis support apparatus shown as the first to fourth embodiments, and is a normal endoscopic image (hereinafter referred to as a normal image) and a medicine distribution. By determining whether the image is an endoscopic image (hereinafter referred to as a stained image) and appropriately selecting an image to be processed in the feature amount calculation from each of the RGB images, it is possible to always provide good diagnosis support information. The present invention relates to an endoscopic diagnosis support apparatus that can be provided.

  The configuration of the endoscope diagnosis support apparatus according to the present embodiment is the same as that of the endoscope diagnosis support apparatus described in the first to fourth embodiments. Therefore, different points will be described below.

  18 to 20 relate to the fifth embodiment of the present invention, FIG. 18 is a block diagram for explaining the configuration of the diagnosis support processing execution program, and FIG. 19 is for explaining the operation of the diagnosis support processing execution program. FIG. 20 is an explanatory diagram for explaining a determination process for a normal image and a stained image.

  As shown in FIG. 18, the diagnosis support processing execution program 250 includes an image input / management block 251, a database management block 252, an image type determination block 253, a temporary ROI setting block 254, a feature amount calculation block 255, and an ROI setting block 256. And a classification block 257, a report creation block 258, and an image processing block 239. Among these, the blocks other than the image type determination block 253 perform the same processing as that of each block in the diagnosis support processing execution program 80 described in the first embodiment.

  In the image type determination block 253, the following series of processing is applied to the input endoscopic image. Here, the case where three types of drugs, indigo carmine, methylene blue and crystal violet are selectively used will be described.

  In the present embodiment, steps S20 and S21, which are operations of the diagnosis support process execution program 80 shown in the first embodiment described with reference to FIG. 7, are replaced with the processes shown in steps S100 to S107 in FIG. change.

  In step S100, an endoscope image to be processed is input. Next, in step S101, an image type determination process described later is performed.

  In step S102, it is determined whether the image type determination result is a normal image. If Yes, the process proceeds to step S106, and if No, the process proceeds to step S103.

  In step S103, it is determined whether or not the image type determination result is a stained image by crystal violet. If the determination result is Yes, the process proceeds to step S106, and if the determination result is No, the process proceeds to step S104.

  In step S104, it is determined whether the image type determination result is a stained image of methylene blue. If the determination result is Yes, the process proceeds to step S107, and if the determination result is No, the process proceeds to step S105.

  In step S105, it is determined whether the image type determination result is a stained image of indigo carmine. If the determination result is Yes, the process proceeds to step S107, and if the determination result is No, the process proceeds to step S106. When it is determined No in step S105, there is a possibility that the image is a stained image using another medicine. In this embodiment, a G image is selected at this time.

  In step S106, it applies to the process after step S21 shown in FIG. 7 with respect to G image among RGB images, and progresses to step S22.

  In step S107, it applies to the process after step S21 shown in FIG. 7 with respect to R image among RGB images, and progresses to step S22.

  Next, the processing content of the image type determination in step S101 will be described. In the present embodiment, the image type is determined according to the overall color tone of the endoscopic image.

  Let the pixel values of the RGB image in the input endoscopic image be (ri, gi, bi). Here, i is a continuous number assigned to each surface element and is an integer of 1 or more. For example, when the size of the image is X × Y and all the pixels are used for the determination, 1 ≦ i ≦ X × Y. Further, when M pixels are used by sampling or the like, 1 ≦ i ≦ M. Hereinafter, it is assumed that M pixels are used.

From each surface element, Si = tan ⁻¹ (gi / ri) and Ti = tan ⁻¹ (bi / ri) are obtained. Since ri, gi, and bi have values of 0 or more (for example, 0 to 255 in an 8-bit gradation), the values of Si and Ti are real values in the range of 0 ≦ Si and Ti <90. Actually, an approximate value obtained by dividing the range by 90 is used. For example, it can be discretized into 0, 1, 2,..., 89 by rounding off at the first decimal place.

Si and Ti are numerical values defined by the ratio of the values (ri, gi, bi) of each surface element, and the distribution of values obtained in the normal image and various dyed images is different. Specifically, a normal image with a large value of ri is relatively small for both Si and Ti, and indigo carmine and methylene blue, which are blue drugs, have a lower value of ri compared to the normal image, so both Si and Ti are large. Further, since m i is lower than methylene blue when compared with indigo carmine, T i> S i. On the other hand, crystal violet, which is a purple drug, has a large value of ri and bi with respect to a value of gi, so that Si is small and Ti is large. Figure 20 shows the difference between these distributions.
In FIG. 20, (1) is a normal image, (2) is an indigo carmine stained image, (3) is a methylene blue stained image, and (4) is a region where many pixels are distributed in the crystal violet stained image. Yes. With this property, for example, when 60% or more of M pixels are included, it can be determined that the image type corresponds to the corresponding region. In addition, in the case where it is impossible to determine any image type such as a distribution that is very rare, the image is determined as an unknown image.

  As described above, according to the endoscopic diagnosis support apparatus in the present embodiment, it is determined whether the endoscopic diagnosis support apparatus is a normal endoscopic image or an endoscopic image in which a medicine is dispersed. It is possible to determine and appropriately select an image to be processed in the feature amount calculation from the RGB images, and it is possible to always provide good diagnosis support information.

  Of course, the operator can manually determine the image type.

  When the determination is impossible, the determination may be made again after increasing the number of samples of the pixels used for the determination or changing the position.

Sixth embodiment:
The sixth embodiment of the present invention relates to the endoscope diagnosis support apparatus shown as the first to fifth embodiments, determines whether it is a normal image or a stained image, and according to the result The present invention relates to an endoscopic diagnosis support apparatus that can provide good diagnosis support information by making it possible to use a mixture of images by performing gradation inversion.

  The configuration of the endoscope diagnosis support apparatus and the configuration block of the diagnosis support process execution program in the present embodiment are the same as those in the endoscope diagnosis support apparatus described in the fifth embodiment. Therefore, different points will be described below.

  FIG. 21 is a flow chart for explaining the operation of the diagnosis support process execution program according to the sixth embodiment of the present invention.

  In the stained image, the appearance of the mucosal surface structure on the image is reversed due to the difference in the properties of the drugs used as shown in FIG. For example, in the observation of the shape of the ostium, the stained image using indigo carmine has a lower pixel value with respect to the peripheral part in the stained part, so that it is normally absorbed by the living body such as methylene blue and crystal violet. Therefore, it is different from a dye having a property of obtaining an effect. Therefore, for example, when applying a feature value calculation method with binarization, the processing results are different, so that inversion of gradation is necessary.

  FIG. 21 is an explanatory diagram for explaining the operation of the diagnosis support processing execution program 250 in the present embodiment. In the present embodiment, steps S20 and S21, which are operations of the diagnosis support process execution program 80 shown in the first embodiment described with reference to FIG. 7, are replaced with the processes shown in steps S110 to S120 in FIG. change.

  In FIG. 21, the content of each process shown in steps S110 to S115 is the same as that in steps S100 to S105 of FIG. 19 in the fifth embodiment.

  In step S112, it is determined whether the image type determination result is a normal image. If Yes, the process proceeds to step S116, and if No, the process proceeds to step S113.

  In step S113, it is determined whether or not the image type determination result is a stained image by crystal violet. If the determination result is Yes, the process proceeds to step S116, and if the determination result is No, the process proceeds to step S114.

  In step S114, it is determined whether or not the image type determination result is a methylene blue stained image. If the determination result is Yes, the process proceeds to step S117. If the determination result is No, the process proceeds to step S115.

  In step S115, it is determined whether the image type determination result is a stained image of indigo carmine. If the determination result is Yes, the process proceeds to step S119, and if the determination result is No, the process proceeds to step S118. If it is determined No in step S115, there is a possibility that the image is a stained image using another medicine. In this embodiment, a G image is selected at this time.

  In steps S116 and S118, the G image is selected as a subsequent processing target image, and the process proceeds to step S21.

  In step S117, the R image is selected as a subsequent processing target image, and the process proceeds to step S21.

  In step S119, the R image is selected as a subsequent processing target image, and the process proceeds to step S120.

  In step S120, tone inversion processing is applied to enable mixed use of indigo carmine and other types of images. In the gradation inversion processing, the image processing block 259 performs pixel value conversion using the following expression.

r'j = 255-rj
Here, rj is the value of pixel j in the R image (1 ≦ j ≦ X × Y, where X and Y are the vertical and horizontal sizes of the image), is the number of gradations of 8 bits, and 0 ≦ rj ≦ 255. The range shall be taken. r′j is the value of the pixel after gradation inversion and is used for the subsequent processing. After applying gradation inversion processing to the R image in step S120, the process proceeds to step S21.

  As described above, according to the endoscope diagnosis support apparatus in the present embodiment, each image is determined by determining whether the image is a normal image or a stained image, and performing inversion of gradation according to the result. Can be used in combination, and good diagnostic support information can be provided.

Seventh embodiment:
The seventh embodiment of the present invention relates to the endoscopic diagnosis support apparatus shown as the first to fifth embodiments, which is either a stained image using indigo carmine or any other endoscopic image. Endoscopic diagnosis support that can provide good diagnosis support information by determining whether or not and by using each image mixed by inverting the gradation according to the result It relates to the device.

  In the sixth embodiment of the present invention, it is determined whether the input endoscopic image is a normal image or three types of stained images, and in consideration of the difference in properties of the used drug, An endoscopic diagnosis support apparatus has been described in which an R or G image is selected and tone reversal processing is applied to a stained image using indigo carmine.

  On the other hand, as described above, indigo carmine contains many structural components of the mucosal surface in the R image due to its blue-green color tone, but the structure sufficient for calculation of the feature amount also in the G image due to the nature of the original endoscopic image. In such a case, the G image can be used in common.

  In the present embodiment, the G image is commonly used in applying the feature amount calculation method from various endoscope images, and the gradation conversion process is applied only when the image is a stained image using indigo carmine. An endoscope diagnosis support apparatus that simplifies the processing in the sixth embodiment and saves memory resources and speeds up the processing will be described.

  The configuration of the endoscope diagnosis support apparatus and the configuration block of the diagnosis support processing execution program in this embodiment are the same as those in the endoscope diagnosis support apparatus described in the fifth embodiment. Therefore, different points will be described below.

  FIG. 22 is a flowchart for explaining the operation of the diagnosis support processing execution program according to the seventh embodiment of the invention.

  In the present embodiment, steps S20 and S21, which are operations of the diagnosis support process execution program 80 shown in the first embodiment described with reference to FIG. 7, are replaced with the processes shown in steps S140 to S144 in FIG. change.

  In FIG. 21, the content of each process shown in steps S140 and S141 is the same as that in steps S100 and S101 of FIG. 19 in the fifth embodiment.

  In step S142, a G image is obtained from the input endoscopic image.

  In step S143, it is determined whether the image type determination result is a stained image of indigo carmine. If the determination result is Yes, the process proceeds to step S144, and if the determination result is No, the process proceeds to step S121.

  In step S144, tone reversal processing is applied to enable mixed use of indigo carmine and other types of images. In the gradation inversion processing, the image processing block 259 converts pixel values using the following formula.

g'j = 255-gj
Here, gj is the value of pixel j in the G image (1 ≦ j ≦ X × Y, where X and Y are the vertical and horizontal sizes of the image), is the number of gradations of 8 bits, and 0 ≦ gj ≦ 255. The range shall be taken. g ′ j is the value of the pixel after gradation inversion and is used for the subsequent processing. After applying the gradation inversion process for the G image, the process proceeds to step S21.

  As described above, according to the endoscopic diagnosis support apparatus in the present embodiment, it is determined whether a stained image using indigo carmine or an endoscopic image other than that is used, and according to the result By performing inversion of gradation, it is possible to provide a good diagnosis support information by making it possible to use each image in a mixed manner.

Eighth embodiment:
The eighth embodiment of the present invention relates to the endoscope diagnosis support apparatus shown as the first to seventh embodiments, and in applying the spatial frequency analysis method in the feature amount calculation, the space of the endoscope image The present invention relates to an endoscope diagnosis support apparatus that can obtain highly accurate diagnosis support information by using all information of frequency components.

  The configuration of the endoscope diagnosis support apparatus according to the present embodiment is the same as that of the endoscope diagnosis support apparatus described in the first embodiment. The diagnosis support process execution program in the present embodiment is the same as the diagnosis support process execution program 80 shown in FIG. 6, and the object is achieved by changing the feature amount calculation method applied in the feature amount calculation block 84. To do.

  The spatial frequency analysis method extracts a frequency component by applying filtering to an image and converts it into a feature amount. For example, the following power spectrum method is known as the most general method.

The filtering processing result g (x, y) by the image t (x, y) and the filter f (x, y) is expressed by [Equation 2].
G (u, v) = T (u, v) · F (u, v) (2)
It is represented by Here, G (u, v), T (u, v), and F (u, v) are Fourier transforms of g (x, y), t (x, y), and f (x, y), respectively. . The filter f (x, y) has a low-frequency, high-frequency, or band-pass type frequency characteristic, and has an effect of extracting a specific frequency component in the image. Also, Equation (2) is calculated by [Equation 3] in the real space by the convolution operation of the image and the filter.
g (x, y) = t (x, y) * f (x, y) (3)
Generally, it is executed by digital filtering using an FIR filter or the like.

［数７］
ψ（ｕ，ｖ）＝ｔａｎ^-1｛Ｉｍ（Ｇ（ｕ，ｖ））／Ｒｅ（Ｇ（ｕ，ｖ）｝ （７）
と表される。 G (u, v) is a complex number, and its real and imaginary terms are Re (G (u, v)) and Im (G (u, v)), respectively [Equation 4]
G (u, v) = Re (G (u, v)) + jIm (G (u, v)) (4)
It is expressed. Here, j represents an imaginary unit. Furthermore, from Equation (4), G (u, v) is expressed by using the amplitude term A (u, v) and the phase term ψ (u, v) [Equation 5]
G (u, v) = A (u, v) exp (jψ (u, v)) (5)
[Equation 6]

[Equation 7]
ψ (u, v) = tan ⁻¹ {Im (G (u, v)) / Re (G (u, v)} (7)
It is expressed.

  In the power spectrum method, it is obtained from n (f is an integer greater than or equal to 1) filters f (x, y) to which the amplitude term A (u, v) shown in Equation (6) is applied and used as a feature quantity It is.

  In addition, the Gabor feature widely used in the field of image analysis in recent years is a frequency characteristic of the filter f (x, y) set by a Gabor function considering the characteristics of the human visual system. Similar to the spectrum method, the amplitude term A (u, v) is used as the feature quantity.

  As shown in Expression (5), the image includes an amplitude term A (u, v) and a phase term ψ (u, v) as information on frequency components, but the latter is not used as a feature quantity, and thus information. Loss will occur. In the present embodiment, in addition to the feature quantity based on the amplitude term A (u, v), the feature quantity based on the phase term ψ (u, v) is calculated, thereby improving the accuracy of the diagnosis support information.

［数９］

で定義される。大きさＮ×Ｎの画像ｔとＧａｂｏｒフィルタｆとの畳み込み演算による処理結果ｇは，画素ｔ（Ｘ，Ｙ）に対して、
［数１０］

で与えられる。 In the present embodiment, it is assumed that K × M Gabor filters are used to extract frequency components from an endoscopic image. K and M are the number of parameters to be described later that define the Gabor filter. The Gabor filter is a product of a two-dimensional Gaussian curved surface and a plane wave propagating in one direction on a two-dimensional plane. The Gabor filter has a standard value difference σ _x and σ _{y on the} Gaussian curved surface, the plane wave traveling direction θ _k and the wavelength λ _m It is determined. Here, k and m are parameters of the traveling direction and wavelength that define the Gabor filter, respectively, and 0 ≦ k <K and 0 ≦ m <M. Standard value difference sigma _x and sigma _y is closely related to the wavelength lambda _m, can be a function of the wavelength lambda _m, are respectively expressed as σ _x (λ _m) and σ _y (λ _m). The Gabor filter f is a two-dimensional filter composed of a real part Re (g) and an imaginary part Im (g).
[Equation 8]

[Equation 9]

Defined by The processing result g by the convolution operation of the image t of size N × N and the Gabor filter f is the pixel t (X, Y),
[Equation 10]

Given in.

Using g in equation (10), the values of Gabor features hk, m (X, Y) are
[Equation 11]
h _{k, m} (X, Y) = | g _{k, m} (X, Y) | (11)
It becomes. Here, | · | represents the absolute value (α ² + β ² ) ^1/2 of the complex number α + jβ.

  In this embodiment, as a feature quantity using amplitude and phase information, Reference 4 [Rotation-Invariant Texture Classification Using a Complete Space-Frequency Model, GMHaley and BSManjunath, IEEE TRANS. ON IMAGE PROCESSING, Vol.8, No.2, FEB. 1999] is calculated.

［数１３］

［数１４］

［数１５］

［数１６］

［数１７］

ここで，ｎｘ及びｎｙはそれぞれ大きさＩＳＸ×ＩＳＹの画像における画素の座標で、０≦ｎｘ＜ＩＳＸ， ０≦ｎｙ＜ＩＳＹである。また、ｓはＧａｂｏｒフィルタの波長λ_mに対応するパラメタであり、１≦ｓ≦Ｍである。また、ｒはＧａｂｏｒフィルタの方向θ_kに対応するパラメタであり、０≦ｒ＜Ｋである。ここでは表記を文献４に合わせ，Ｒ＝Ｋであるものとする。また、ａｒｇ［・］はｔａｎ^-1｛Ｉｍ（・）／Ｒｅ（・）｝を示す。 [Equation 12]

[Equation 13]

[Formula 14]

[Equation 15]

[Equation 16]

[Equation 17]

Here, nx and ny are pixel coordinates in an image of size ISX × ISY, and 0 ≦ nx <ISX and 0 ≦ ny <ISY. Further, s is a parameter corresponding to the wavelength λ _{m of the} Gabor filter, and 1 ≦ s ≦ M. R is a parameter corresponding to the direction θ _{k of the} Gabor filter, and 0 ≦ r <K. Here, the notation is adjusted to Document 4 and R = K. Further, arg [•] indicates tan ⁻¹ {Im (•) / Re (•)}.

［数２０］

ここで、∇_x（）及び∇_y（）は勾配推定関数、θ_rはＧａｂｏｒフィルタの方向である。また、
［数２１］
θ∇＝ｔａｎ^-1｛▽_y（ψ_s,r（ｎｘ，ｎｙ））／▽_x（ψ_s,r（ｎｘ，ｎｙ）｝ （２１）
より求められる。また、ａｒｇ［・］はａｔａｎ（Ｉｍ（・）／Ｒｅ（・））を示す。 Further, a _{s, r} (nx, ny), φ _{s, r} (nx, ny) and u _{s, r} (nx, ny) are obtained as follows.
[Equation 18]
a _{s, r} (nx, ny) = | g _{s, r} (nx, ny) | (18)
[Equation 19]

[Equation 20]

Here, ∇ _x () and ∇ _y () are gradient estimation functions, and θ _r is the direction of the Gabor filter. Also,
[Equation 21]
θ∇ = tan ⁻¹ {▽ _y (ψ _{s, r} (nx, ny)) / ▽ _x (ψ _{s, r} (nx, ny)} (21)
More demanded. Also, arg [•] indicates atan (Im (•) / Re (•)).

  The feature amounts shown in the equations (# 11) to (# 16) are calculated for each surface element, and can be applied to step S26 in FIG. 7 shown in the first embodiment.

  Furthermore, it is possible to calculate the following feature quantity for each region from the feature quantity for each pixel obtained by the equations (12) to (17).

ここで，ｆA，ｆF，ｆY，ｆDA，ｆDF及びｆDYはそれぞれｆA_s,p，ｆF_s,q，ｆY_s,q，ｆDA_s,q，ｆDF_s,q及びｆDY_s,qに対するｐまたはｑに基づくベクトル表記である。また、Ｅ｛｝は算出対象とする領域内（ＲＯＩ）の各面素に碁づく期待値（平均値），*は複素共役を示す。 [Equation 22]

Where fA , fF , fY , fDA , fDF, and fDY are p or q for fA _{s, p} , fF _{s, q} , fY _{s, q} , fDA _{s, q} , fDF _{s, q,} and fDY _s, q, respectively. Vector notation based on. E {} represents an expected value (average value) associated with each surface element in the region (ROI) to be calculated, and * represents a complex conjugate.

で定義される．式（２３）においてσ_stは
［数２４］

により求められる。ここで、ＮはＲＯＩ内の画素数、μs及びμtはそれぞれｖs及びｖtのＲＯＩ内の平均値を示す。得られた共分散行列Σにおいて、各σ_st をＲＯＩの特徴量として使用する。式（１４）ないし（１７）に示した各特徴量を含め、同様に分散共分散行列を求めることができる。 It is also possible to calculate a feature quantity based on the variance and covariance of each feature quantity. For example, the variance and covariance using fA _{s, p} and fF _{s, q} shown in equations (12) and (13) are calculated as a variance covariance matrix Σ. For example, when s = 1 and R = 8, three fA _{s, p} and fF _{s, q} are calculated for each surface element _, and are assigned to the j-th pixel (coordinates (nx, ny)) in the ROI. V1j = fA _0,0 (nx, ny), v2j = fA _0,1 (nx, ny), v3j = fA _0,2 (nx, ny), v4j = fF _0,0 (nx, ny), If v5j = fF _0,1 (nx, ny) and v6j = fF _0,2 (nx, ny), then the covariance matrix Σ is
[Equation 23]

Is defined by. In Equation (23), σ _st is [Equation 24].

Is required. Here, N is the number of pixels in the ROI, and μs and μt are the average values in the ROI of vs and vt, respectively. In the obtained covariance matrix Σ, each σ _st is used as a feature amount of ROI. A variance-covariance matrix can be obtained in a similar manner including the feature amounts shown in equations (14) to (17).

  It is also possible to obtain a correlation matrix from the covariance matrix Σ and use the correlation coefficient as a feature quantity.

This embodiment can be applied to step S27 in FIG. 7 shown in the first embodiment, and FCA (i) to FDMY (i ) from each ROI (i) after application of the region division processing. ) And covariance (or correlation coefficient) are obtained and used for the discrimination classification process in step S29.

  As described above, according to the endoscope diagnosis support apparatus in the present embodiment, all information of the spatial frequency component of the endoscopic image is used when applying the spatial frequency analysis method in the feature amount calculation. Thus, highly accurate diagnosis support information can be obtained.

[Appendix]
(Additional Item 1) Frequency component extraction means for extracting a predetermined frequency component from an imaging signal obtained by imaging a subject;
Phase information detection means for detecting phase information of the frequency component extracted by the frequency component extraction means;
A diagnostic support apparatus, comprising: feature amount calculation means for calculating a feature amount of the subject based on the phase information detected by the phase information detection means.

(Additional Item 2) Endoscopic image input means for inputting an endoscopic image composed of a plurality of color signals;
First area setting means for setting a first area for the endoscopic image input to the endoscopic image input means;
Feature quantity calculating means for calculating a feature quantity based on at least one color signal of the endoscopic image;
Second area setting means for setting a second area in the first area set by the first area setting means based on the feature quantity calculated by the feature quantity calculation means;
Discriminating and classifying means for applying the discriminating and classifying process based on the feature quantity calculated by the feature quantity calculating means to at least one area set by the second area setting means;
An endoscopic diagnosis support apparatus, comprising: display means for displaying a discrimination classification result obtained by the discrimination classification means.

(Additional Item 3) The endoscope diagnosis support apparatus according to Additional Item 2, wherein the display unit displays the feature amount calculated by the feature amount calculation unit.

(Additional Item 4) The endoscope diagnosis support apparatus according to Additional Item 2 or 3, further comprising an area calculating unit that calculates an area of the second region set by the second region setting unit. .

(Additional Item 5) In the Additional Item 4, the second region includes an area ratio calculation unit that calculates an area ratio, and the display unit displays the area ratio calculated by the area ratio calculation unit. The endoscope diagnosis support apparatus described.

(Additional Item 6) The endoscope diagnosis support apparatus according to Additional Item 4 or 5, wherein the area calculating unit calculates an area based on the number of pixels.

(Additional Item 7) The endoscope according to Additional Item 2, 3, 4, 5, or 6, wherein each of the second region setting unit and the discriminating / classifying unit uses different types of feature quantities. Diagnosis support device.

(Additional Item 8) Additional Item 2, 3, 4, 5 in which the second region setting unit sets the second region by region division based on the feature amount calculated by the feature amount calculation unit. , 6 or 7 of the endoscopic diagnosis support apparatus.

(Supplementary Item 9) The supplementary item 2, 3, 4, 5, 6, 7 or 8, wherein the first region set by the first region setting unit is the entire endoscope image. Endoscopic diagnosis support device.

(Additional Item 10) Comprehensive information deriving means for deriving comprehensive diagnosis support information based on a discrimination classification result by the discrimination classifying means for the second area, and the display means is derived by the comprehensive information deriving means The endoscopic diagnosis support apparatus according to the additional item 2, 3, 4, 5, 6, 7, 8 or 9, wherein the diagnosis support information is displayed.

(Additional Item 11) The comprehensive information deriving unit calculates an area ratio obtained by integrating the second regions for each discrimination classification result, and derives diagnosis support information based on the discrimination classification result and the area ratio. The endoscopic diagnosis support apparatus according to Item 10, wherein the endoscope diagnosis support apparatus is characterized.

(Additional Item 12) The endoscope diagnosis support apparatus according to Additional Item 11, wherein the comprehensive information deriving unit derives diagnosis support information by executing an identification classification process using the area ratio as a feature amount. .

(Additional Item 13) Endoscopic image input means for inputting an endoscopic image composed of a plurality of color signals;
Area setting means for setting an area for the endoscopic image input to the endoscopic image input means;
Endoscopic image type determining means for determining the type of the endoscopic image;
Feature quantity calculating means for calculating a feature quantity based on at least one color signal of the endoscopic image;
Discriminating and classifying means for applying a discriminating and classifying process based on the feature quantity calculated by the feature quantity calculating means to the area set by the area setting means;
Display means for displaying the discrimination classification result by the discrimination classification means,
An endoscope diagnosis support apparatus, wherein the feature amount calculation unit changes a color signal for calculating a feature amount based on a determination result of the endoscope image type determination unit.

(Additional Item 14) An endoscope image type determining unit that determines the type of the endoscope image input to the endoscope image input unit is provided, and based on a determination result of the endoscope image type determining unit Endoscopic diagnosis support apparatus according to additional clause 2, 3, 4, 5, 6, 7, 9, 10, 11 or 12, wherein the feature amount calculation means changes the color signal for calculating the feature amount .

(Additional Item 15) The endoscope diagnosis support apparatus according to Additional Item 13 or 14, wherein the endoscope image type determination unit determines the type of the endoscope image based on the presence / absence of drug dispersion.

(Additional Item 16) The endoscope diagnosis support apparatus according to Additional Item 15, wherein the endoscope image type determination unit determines the type of image based on the type of medicine dispersed.

(Additional Item 17) Additional Item 13, wherein the feature amount calculation unit corrects a color signal for calculating a feature amount based on the determination result of the endoscope image type determination unit. The endoscopic diagnosis support apparatus according to 14, 15 or 16.

(Additional Item 18) The endoscope diagnosis support apparatus according to Additional Item 17, wherein the color signal correction unit performs correction based on inversion of gradation.

(Additional Item 19) The additional image item 13, 14, 15, 16, 17 or 18 in which the endoscope image type determination unit determines the type of the endoscope image based on the color tone of the endoscope image. The endoscopic diagnosis support apparatus according to 1.

(Additional Item 20) Additional Item 2, 3, 4, 5, 6, 7, 9, 10, wherein the feature amount calculating means calculates a feature amount based on a spatial frequency component of the endoscopic image. The endoscope diagnosis support apparatus according to 11, 12, 13, 14, 15, 16, 17, 18, or 19.

(Additional Item 21) The endoscope diagnosis support apparatus according to Additional Item 20, wherein the feature amount calculating unit calculates a feature amount based on an amplitude and / or a phase in a spatial frequency component of the endoscopic image. .

(Additional Item 22) Additional Item 2, 3, 4, 5, 6, 7, 8, 9 wherein the discriminant classification means applies classification using at least one statistical or non-statistical classifier. , 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 or 21.

(Additional Item 23) Endoscopic image input means for inputting an endoscopic image composed of a plurality of color signals;
Spatial frequency component extraction means for extracting a spatial frequency component based on at least one color signal of the endoscopic image input to the endoscopic image input means;
Phase component detection means for detecting a phase component of the endoscopic image based on the spatial frequency component extracted by the spatial frequency component extraction means;
An endoscope image processing apparatus comprising: a feature amount calculating unit that calculates a feature amount based on the phase component detected by the phase component detecting unit.

(Additional Item 24) The endoscopic image processing device according to Additional Item 23, wherein the feature amount calculation method calculates a feature amount based on a change direction of the phase component.

(Additional Item 25) The endoscopic image processing device according to Additional Item 23 or 24, wherein the feature amount calculation method calculates a feature amount based on a magnitude of a change in the phase component.

(Additional Item 26) Amplitude component detection means for detecting an amplitude component of the endoscopic image based on the spatial frequency component extracted by the spatial frequency component extraction means;
A feature amount calculating unit that calculates a feature amount based on the phase component detected by the phase component detecting unit and the amplitude component detected by the amplitude component detecting unit;
An endoscopic image processing device according to Additional Item 23, 24 or 25, comprising:

(Additional Item 27) The endoscopic image processing device according to Additional Item 26, wherein the feature amount calculating unit calculates a feature amount based on a covariance or correlation of the phase component and the amplitude component.

(Additional Item 28) The endoscopic image processing device according to Additional Item 23, 24, 25, 26, or 27, wherein the spatial frequency component extracting unit performs Fourier transform or filtering using a digital filter.

(Additional Item 29) A step of inputting an endoscopic image;
Applying filtering to extract a spatial frequency component on the endoscopic image;
Detecting phase information of the endoscopic image based on the application result of the filtering;
An endoscopic image processing method comprising: calculating a feature amount based on the phase information.

(Additional Item 30) Endoscopic image input means for inputting an endoscopic image composed of a plurality of color signals, and area setting means for setting an area for the endoscopic image input to the endoscopic image input means When,
Endoscopic image type determining means for determining the type of the endoscopic image;
A feature amount calculating means for calculating a feature amount based on at least one color signal of the endoscopic image,
An endoscope image processing apparatus, wherein the feature amount calculation unit changes a color signal for calculating a feature amount based on a determination result of the endoscope image type determination unit.

(Additional Item 31) Inputting an endoscope image composed of a plurality of color signals;
Setting an area for the endoscopic image;
Determining the type of the endoscopic image;
Calculating a feature amount based on at least one color signal of the endoscopic image,
An endoscopic image processing method, comprising: changing a color signal for calculating a feature amount in the feature amount calculation step based on a determination result of the endoscope image type determination unit.

  The purpose of supplementary items 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, and 22 is when the mucosal surface structure of the lesion shows a plurality of different findings in the endoscopic image, or the lesion Even when other different tumors are mixed, a region of interest is set for each site showing each finding, and what kind of findings or tumors are diagnosed in relation to the findings. Furthermore, it is providing the diagnostic assistance apparatus which can know what ratio those are mixed.

  Additional Item 1 3,1 4,1 5,1 6,1 7,1, 8, 19, 30 and 31 are used to determine which of the RGB images is to be processed in the feature quantity calculation, The objective is to make an objective and numerical diagnosis with higher accuracy, depending on the type of drug used for spraying.

  The purpose of the additional items 20, 22, 23, 24, 25, 26, 27, 28, and 29 is to apply a feature quantity calculation method that takes phase information into consideration in the feature quantity of the spatial frequency component in the endoscopic image. The objective is to make more accurate objective and numerical diagnosis using all the information of spatial frequency components.

The block diagram which shows the structure of the endoscope diagnosis assistance apparatus which concerns on the 1st Embodiment of this invention. Functional block diagram for explaining the configuration of a diagnosis support processing execution program in a conventional endoscope diagnosis support apparatus Flowchart for explaining the operation of a diagnosis support process execution program in a conventional endoscope diagnosis support apparatus ROI setting window explanatory diagram for explaining ROI setting in a conventional endoscope diagnosis support apparatus Report display window explanatory diagram for explaining display of diagnosis support information in a conventional endoscope diagnosis support apparatus Functional block diagram for demonstrating the structure of the diagnostic assistance process execution program in the endoscope diagnostic assistance apparatus of 1st Embodiment The flowchart for demonstrating operation | movement of the diagnosis assistance process execution program in the endoscope diagnosis assistance apparatus of 1st Embodiment. Temporary ROI setting window explanatory drawing for demonstrating temporary ROI setting in the endoscope diagnosis assistance apparatus of 1st Embodiment Flowchart for explaining the operation of area division processing according to the first embodiment 1st explanatory drawing for demonstrating the area division | segmentation process of 1st Embodiment 2nd explanatory drawing for demonstrating the area | region division process of 1st Embodiment Report display window explanatory drawing for demonstrating the display of the diagnostic assistance information in the endoscope diagnostic assistance apparatus of 1st Embodiment The block diagram for demonstrating the structure of the diagnosis assistance process execution program in this Embodiment which concerns on the 2nd Embodiment of this invention. The flowchart for demonstrating operation | movement of the diagnostic assistance execution program of 2nd Embodiment Flowchart for explaining derivation of diagnosis support information according to the second embodiment Report display window explanatory drawing for demonstrating the display of the diagnostic assistance information in the endoscope diagnostic assistance apparatus of 2nd Embodiment Report display window explanatory drawing for demonstrating the display of the diagnostic assistance information in the endoscope diagnostic assistance apparatus which concerns on the 3rd Embodiment of this invention. The block diagram for demonstrating the structure of the diagnostic assistance process execution program which concerns on the 5th Embodiment of this invention The flowchart for demonstrating operation | movement of the diagnostic assistance process execution program in 5th Embodiment Explanatory drawing for demonstrating the determination process of the normal image and dyed | stained image in 5th Embodiment Explanatory drawing for demonstrating operation | movement of the diagnostic assistance process execution program 250 which concerns on the 6th Embodiment of this invention. The flowchart for demonstrating operation | movement of the diagnostic assistance process execution program which concerns on the 7th Embodiment of this invention Explanatory drawing for explanation about dyed image

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Diagnosis support apparatus 2 ... Video processor 3, 35 ... Observation monitor 4 ... Input unit 5 ... Server unit 6 ... Conference unit 11 ... A / D converter 12, 32, 50 ... Image processing part 13, 21, 31L ... LAN controller 14, 26, 36 ... Controller 22 ... Memory 23, 47 ... Hard disk 24 ... Hard disk driver 25 ... Compression device 33 ... Expansion device 34 ... D / A converter 41 ... CPU
42 ... Keyboard 43 ... Keyboard I / F
44 ... Search monitor 45 ... Mouse 46 ... Mouse I / F
48 ... Hard disk I / F
49 ... Work memory 50 ... Image processing unit 51 ... Printer 52 ... Printer I / F
61 ... Password storage unit 62 ... Password monitoring unit 63 ... Control restriction unit 80 ... Diagnosis support processing execution program 81 ... Image input / management block 82 ... Database management block 83 ... Temporary ROI setting block 84 ... Feature quantity calculation block 85 ... ROI setting Block 86 ... Discrimination classification block 87 ... Report creation block 88 ... Image processing block

Claims (10)

Frequency component extraction means for extracting a predetermined frequency component from at least one imaging signal among a plurality of imaging signals obtained by imaging the subject;
Phase information detection means for detecting phase information based on the predetermined frequency component extracted by the frequency component extraction means;
Feature quantity calculating means for calculating the feature quantity of the subject based on the phase information detected by the phase information detecting means;
Feature quantity calculation means for calculating the feature quantity calculated by the feature quantity calculation means;
A diagnostic support apparatus comprising:   The endoscope diagnosis support apparatus according to claim 1, wherein the feature amount calculation unit calculates a feature amount based on a spatial frequency component of the endoscopic image.   The endoscope diagnosis support apparatus according to claim 2, wherein the feature amount calculation unit calculates a feature amount based on an amplitude and / or a phase in a spatial frequency component of the endoscopic image. Endoscopic image input means for inputting an endoscopic image composed of a plurality of color signals;
Spatial frequency component extraction means for extracting a spatial frequency component based on at least one color signal of the endoscopic image input to the endoscopic image input means;
Phase component detection means for detecting a phase component of the endoscopic image based on the spatial frequency component extracted by the spatial frequency component extraction means;
An endoscope image processing apparatus comprising: a feature amount calculating unit that calculates a feature amount based on the phase component detected by the phase component detecting unit.   The endoscopic image processing apparatus according to claim 4, wherein the feature amount calculation method calculates a feature amount based on a change direction of the phase component.   The endoscopic image processing apparatus according to claim 4, wherein the feature amount calculation method calculates a feature amount based on a magnitude of the change in the phase component. Amplitude component detection means for detecting an amplitude component of the endoscopic image based on the spatial frequency component extracted by the spatial frequency component extraction means;
A feature amount calculating unit that calculates a feature amount based on the phase component detected by the phase component detecting unit and the amplitude component detected by the amplitude component detecting unit;
The endoscopic image processing apparatus according to claim 4, further comprising:   The endoscopic image processing apparatus according to claim 7, wherein the feature amount calculation unit calculates a feature amount based on a covariance or correlation between the phase component and the amplitude component.   The endoscope image processing apparatus according to claim 4, wherein the spatial frequency component extraction unit performs filtering using Fourier transform or a digital filter. Inputting an endoscopic image comprising a plurality of color signals;
Applying filtering for extracting a spatial frequency component to the inputted endoscopic image;
Detecting phase information of the endoscopic image based on the application result of the filtering;
Calculating a feature amount based on the detected phase information;
An endoscopic image processing method comprising:

Images