JP2002165757A - Diagnostic supporting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a means for knowing a kind of findings or relevant findings of tumors, or the like, to be diagnosed thereof, and their ratio involved therein. SOLUTION: The diagnostic supporting process implementation program 80 consists of image input-control block 81, data base control block 82, provisional ROI set-up block 83, exceptionally small amount calculation block 84, ROI set-up block 85, distinguishing classification block 86, report preparation block 87 and image processing block 88.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a diagnosis support apparatus, and more particularly, to a diagnosis support apparatus characterized in that a type of a lesion is automatically determined and classified based on image data from an endoscope apparatus.

[0002]

2. Description of the Related Art In recent years, an elongated insertion portion has been inserted into a body cavity,
2. Description of the Related Art An endoscope apparatus capable of observing an organ in a body cavity or the like on a monitor screen using a solid-state imaging device or the like as an imaging unit on a monitor screen to perform inspection or diagnosis is widely used. Further, there is also an ultrasonic endoscope apparatus which irradiates ultrasonic waves to the internal organs of the body cavity, observes the condition of the internal organs of the body cavity on a monitor screen based on the reflection or transmittance of the ultrasonic waves, and can perform inspection or diagnosis. Widely used.

[0003] The final diagnosis using these endoscope devices largely depends on the subjectivity of the doctor, and an endoscope diagnosis support device directly linked to objective and numerical diagnosis has been realized. Was desired.

[0004] An endoscope diagnosis support apparatus uses various feature amounts calculated from a region of interest (ROI) in an image,
Supporting objective and numerical diagnosis by presenting to doctors what kind of findings and lesions are classified into images to be diagnosed using dark value processing or statistical / non-statistical classifiers Is what you do.

[0005] The feature amount is a numerical value reflecting various findings on an endoscope image, and is obtained by applying an image processing technique. For example, in the case of characterizing a finding relating to a color tone such as “red mucous membrane surface is reddened”, R / (R + G
+ B) can be obtained for each pixel, and the average value can be used as a feature value (this feature value is generally called chromaticity). In recent years, in the field of endoscopes, a characteristic quantity of color tone that reflects gastric mucosal blood flow rate is 32 log ₂ (R / G).
Is widely used.

Further, there are various diseases related to the mucosal surface structure in endoscopic images, such as dilatation and meandering of blood vessels and small and large irregularities in the form of small stomach sections, and groove widths of small stomach sections in the see-through blood vessel images. Is an important element in the diagnosis of, and these can also be quantified as feature values by applying an image processing method. In such feature amount calculation, there is a method disclosed in Japanese Patent No. 2918162.

[0007] Further, by using each feature amount obtained from a plurality of different findings in combination as a feature vector, more complex and accurate diagnosis can be supported.
In improving the accuracy of the endoscope diagnosis support device, it can be said that a highly accurate feature amount calculation method for quantifying important endoscopic image findings is very important.

In recent years, the fineness of the mucosal surface structure and the mucosal surface structure have been improved by a spatial frequency analysis technique in which Gabor features calculated using a known Gabor filter are improved for application to endoscopic images. There is an endoscope image processing method in which the directionality of a pattern presented by a structure is quantified as a feature amount.

Japanese Patent Application Laid-Open No. 10-14864 discloses an example of such an endoscope diagnosis support apparatus and a feature amount calculation method.

[0010]

In recent endoscope apparatuses, high image quality, high pixel density of a solid-state imaging device (CCD, etc.), and an outer diameter and operability equivalent to those of a normal endoscope are required. With the advent of a magnifying endoscope that has a magnifying observation function (zooming) while maintaining it, extremely fine capillary images on the mucous membrane surface and stomach and
The pit structure of the large intestine is clearly observed. A biological image of the same level as a conventional tissue specimen observed under a stereoscopic microscope can be observed at the time of clinical examination under an endoscope, and a new diagnostic science using those microstructure observation findings is established. Have been.

[0011] For example, reference 1 (analysis of stereoscopic microscopic images of gastric protuberance lesions and magnified electronic endoscopic images from the histopathological findings, Toru Honda et al., Gastroenterological Endosc
opy, May 1993, Vol. 35, No. 5, pp. 967-976), Reference 2 (Depressed type early colorectal cancer, Shinhide Kudo, Mannebros Gastro, May 1993, Vol. 3, No. 5, pp. 47-) 53) and Reference 3 (Diagnosis of colorectal disease using an enlarged electronic scope, Shindo Kudo et al., Stomach and Intestine, 1994, Vol. 29, No. 3, pp. 163-165).
These indicate that lesions such as normal mucous membranes, benign tumors, and cancers can be distinguished from differences in pit morphology and arrangement observed on endoscopic images. For example, in Literatures 2 and 3, differences in morphology and arrangement of pits found in colorectal lesions are classified into pit patterns composed of type I to type V and subclasses thereof, and associated with various lesions.

However, in realizing an objective and numerical diagnosis for these new endoscopic diagnostics,
The diagnosis support apparatus disclosed in Japanese Patent Application Laid-Open No. H10-14864 has the following problems.

[0013] In endoscopic images, particularly at the time of close-up or magnified observation, not only the cases where the lesions do not necessarily show uniform findings, but also, for example, the extremely small lesions of several mm in the colon pit pattern are normal. In some cases, the type I, the type IIIs of the neoplastic lesion, and the type intermediate between them may be mixed. In addition, lesions in which some of the benign tumors have become cancerous, such as intra-adenoma cancers, have also been reported.

Since information on a region not set as a region of interest cannot be obtained, for example, what type of pit pattern is present on the mucosa of the lesion and at what ratio
Another problem is that it is not possible to know what kind of tumor is mixed in the lesion.

Further, in calculating the feature amount from the endoscope image, it is necessary to use more information in the endoscope image in order to realize more accurate diagnosis. On the other hand,
In Japanese Patent Application Laid-Open No. 0-14864, the above-described spatial frequency analysis methods are applied, but these methods do not use all information of the spatial frequency components. More specifically, the information included in the spatial frequency component includes amplitude information and phase information that constitute an endoscope image, but there is a loss of information due to not using phase information conventionally.

Further, Japanese Patent No. 2918162 discloses calculation of a feature amount of a fine structural component on a mucosal surface, but has the following problem when used in an endoscope diagnosis support apparatus.

In a clinical examination using an endoscope, various pigments such as indigo carmine, methylene blue, and crystal violet, and drugs called staining agents are used to clearly observe the mucosal surface structure. For example, in the observation of the pit pattern of the large intestine, indigo carmine is used as a standard, and when more detailed observation is required, methylene blue or crystal violet is used.

However, indigo carmine used as a dye method and methylene blue and crystal violet used as a dyeing method have the following differences in properties.

Indigo carmine does not have the property of being absorbed by a living body, and a clear observation image can be obtained by accumulating in a concave portion according to the uneven shape of the mucous membrane surface. On the other hand, methylene blue and crystal violet are absorbed into, for example, the nucleus in a cell, thereby making it possible to clarify the contrast with a site not absorbed.

Differences that appear on an image from these properties will be described with reference to FIG. 23 as an example. For example, pit pattern of colon
Is to observe the shape and arrangement of the pits on the mucosal surface, but because the pits are fine holes opened on the mucosal surface,
When indigo carmine is sprayed, the gland orifice is observed as a blue-green area in the endoscopic image. When this is schematically represented, it is a region shown in black in FIG.
On the other hand, in the case of methylene blue and crystal violet, the pores are naturally not stained because they are empty spaces, and as shown in FIG.
In the endoscope image, portions other than the gland orifice are observed as blue regions in methylene blue and purple regions in crystal violet.

Therefore, for example, when the mucosal structure is extracted by threshold processing, the processing result is reversed due to the difference of each drug, and different values of characteristic amounts are calculated despite the same disease. For this reason, there is a problem that a normal image having no medicine sprayed and an image having various medicines sprayed cannot be used in combination.

In a normal endoscopic image without drug spraying, the structural components on the mucous membrane surface are represented by G in the RGB image.
Most included in the image. The same applies to crystal violet because of its purple color tone. On the other hand, when blue-based chemicals such as indigo carmine and methylene blue are used, the R image may contain the most structural components.
Therefore, although which of the RGB images is to be processed in the feature amount calculation is changed in accordance with the presence or absence of medicine spraying and the type of confectionery used for spraying, the accuracy of diagnosis support can be improved. There is a problem that this point is not taken into account.

The present invention has been made in view of the above circumstances, and is directed to a case where a mucosal surface structure of a lesion shows a plurality of different findings in an endoscopic image, or a case where another different tumor or the like is mixed in a lesion. In, in addition to setting a region of interest for each part showing each finding, what kind of findings,
Alternatively, it is an object of the present invention to provide a diagnosis support apparatus that can determine whether a tumor or the like to be diagnosed in association with a finding exists, and further, what ratio they are mixed.

[0024]

According to the present invention, there is provided a diagnostic support apparatus comprising: a feature amount calculating unit configured to calculate a feature amount based on an image signal of an object to be imaged; Image region extracting means for extracting an area of an image formed by the imaging signal based on a plurality of feature amount data corresponding to each pixel; and the plurality of feature amount data corresponding to an image region extracted by the image region extracting means And a display means for displaying the result of the classification by the classification and classification means.

[0025]

Embodiments of the present invention will be described below with reference to the drawings.

First Embodiment: In a first embodiment of the present invention, a region of interest (hereinafter, referred to as an ROI) can be set without exception for a lesion for which acquisition of diagnosis support information is desired. The present invention relates to an endoscope diagnosis support apparatus that is capable of presenting what kind of findings and what percentage of each ROI is present, as well as being possible.

FIGS. 1 to 12 relate to a first embodiment of the present invention. FIG. 1 is a block diagram showing a configuration of an endoscope diagnosis support device, and FIG. 2 is a diagram showing a diagnosis in a conventional endoscope diagnosis support device. FIG. 3 is a functional block diagram for explaining the configuration of a support processing execution program, FIG. 3 is a flowchart for explaining the operation of a diagnosis support processing execution program in a conventional endoscope diagnosis support apparatus, and FIG. FIG. 5 is an explanatory view of an ROI setting window for explaining the ROI setting in the support apparatus, FIG. 5 is an explanatory view of a report display window for explaining the display of diagnosis support information in a conventional endoscope diagnosis support apparatus, and FIG. And FIG. 7 is a functional block diagram for explaining a configuration of a diagnosis support processing execution program in the endoscope diagnosis support apparatus according to the embodiment. FIG. 8 is a flowchart for explaining the operation of the support processing execution program. FIG. 8 is a diagram illustrating a temporary ROI setting window for explaining a temporary ROI setting in the endoscope diagnosis support apparatus according to the present embodiment. 10 and FIG. 11 are explanatory diagrams for explaining the area dividing process in the present embodiment, and FIG. 12 is a flowchart for explaining the operation of the endoscope diagnosis supporting apparatus in the present embodiment. It is a report display window explanatory view for explaining the display of diagnosis support information.

First, the configuration of the endoscope diagnosis support apparatus according to the first embodiment of the present invention will be described with reference to FIG. Note that the configuration of the endoscope diagnosis support apparatus according to the present embodiment is basically the same as that of a conventional endoscope diagnosis support apparatus, but by differentiating the blocks and operations constituting a diagnosis support program described later. It achieves its purpose.

As shown in FIG. 1, a diagnosis support apparatus 1 according to the present embodiment includes, for example, a video processor 2 that obtains an image signal from an electronic endoscope (not shown) and converts it into a video signal. An observation monitor 3 for projecting a video signal, an input unit 4 for converting a video signal from the video processor 2 into image data for signal processing, image data signal-processed by the input unit 4, and lossless or irreversible compression of this image data A server unit 5 for storing the compressed image data, and a search and display of the image data or the compressed image data stored in the server unit 5, and a series of processes such as a region of interest setting process, a feature amount calculation process, and a discriminant classification process. And a conference unit 6 for performing a diagnosis support process.

The input unit 4 is an A / D converter 1 for converting an analog RGB video signal as a video signal from the video processor 2 into image data as a digital signal.
1, an image processing unit 12 having a memory for storing image data and generating an image file to which management information from the video processor 2 is added, and a LAN (Local Area Network) cable The LAN controller 13 transmits the image data to the server unit 5 via the LAN 4a. The controller 14 communicates image data management information with the video processor 2 and controls the image processing unit 12 and the LAN controller 13.

The video processor 2 has a video signal output terminal,
An analog RGB video signal output terminal and a communication signal output terminal are provided. The video signal output terminal is connected to the observation monitor 3, the analog RGB video signal output terminal is connected to the input terminal of the A / D converter 11, and the communication signal output terminal is connected. The end is connected to the controller 14.

The output terminal of the A / D converter 11 is
It is connected to the data signal end of the image processing unit 12.

The control signal end and the data signal end of the controller 14 are connected to the image processing unit 12 and the L
It is connected to a control signal terminal with the AN controller 13. Then, the image processing unit 12 and the L
The control of the AN controller 13 is performed by a signal via the bus line 14a.

For example, an image observed by an electronic endoscope (not shown) converted into a video signal by the video processor 2 is
The image is displayed on the observation monitor 3 as an observation image. When the operator of the video processor 2 determines that the above-described observation image needs to be recorded, the video signal described above is converted into an analog RGB video signal by A / A.
This A / D converter 11 is output to the D converter 11
Performs a predetermined quantization process on an analog RGB video signal, converts the analog RGB video signal into a digital RGB video signal,
2 is output as observation image data.

The image processing section 12 stores observation image data input from the A / D converter 11 under the control of the controller 14.

The controller 14 performs various data processing such as reduction processing on the observation image data stored in the image processing section 12, further adds management information to form an image file, and temporarily stores the image file in the image processing section 12. Or LAN
The data is output to the controller 13. Also,
The controller 14 outputs the image file from the image processing unit 12 to the LAN controller 13 at a predetermined timing when the image file subjected to the various data processing is once stored in the image processing unit 12 as described above. ing.

The server unit 5 has the L of the input unit 4
A LAN controller 21 for receiving an image file sent from the AN controller 13, a memory 22 for temporarily storing the image file received by the LAN controller 21, and a large-capacity storage medium for storing the image file received by the LAN controller 21; Hard disk 2
, A compression device 25 for reversibly or irreversibly compressing the image file received by the LAN controller 21 and sending the compressed image data to the hard disk driver 24, a LAN controller 21, a memory 22, and a hard disk driver. And a controller 26 for controlling the compression device 25. The hard disk driver 24 records the image file and the compressed image data on the hard disk 23, and the LAN controller 21
The image file or the compressed image data recorded on the hard disk 23 can be transmitted to the conference unit 6 via a.

The LAN controller 13 of the input unit 4
The end of the LAN cable 4a is connected to the LAN controller 21 of the server unit 5. This LAN cable 4a is a so-called 10baseT or 100baseT cable, and is capable of performing bidirectional data communication of 10 Mbit / sec within a range of 100 m or less using a twisted pair wire. It can be sent and received.

A control signal terminal and a data signal terminal of the controller 26 of the server unit 5 are connected to a control signal terminal of the memory 22, the LAN controller 21, and the hard disk drive 24 by a bus line 26a.

In the server unit 5, the image file from the LAN controller 13 of the input unit 4 is input by the LAN controller 21 via the LAN cable 4a.
The controller 26 temporarily stores in the memory 22 the image file input via the LAN controller 21.
The hard disk drive 24 stores the image file on the hard disk 23, for example.

The conference unit 6 includes a LAN controller 31 for receiving an image file or compressed image data from the LAN controller 21 of the server unit 5 via the LAN cable 5a, and a LAN controller 3
1, an image processing unit 32 for storing the image file or the compressed image data received, an expansion device 33 for expanding the compressed image data stored by the image processing unit 32, and an image processing unit 3
2 performs a dequantization process on the digital image data and the image data decompressed by the decompression device 22 in the stored image file, and converts the image data into an analog RGB video signal.
An A converter 34, an observation monitor 35 for displaying analog RGB video signals converted by the D / A converter 34, and a controller 36 for controlling the image processing unit 32 are provided.

The data signal end of the image processing unit 32 is D / A
The input terminal of the converter 34 is connected, and the output terminal of the D / A converter 34 is connected to the observation monitor 35. A control signal end and a data signal end of the controller 36 are connected to the image processing unit 32 and the LA by a bus line 36a.
It is connected to a control signal terminal with the N controller 31.
A control signal end and a data signal end of the controller 36 are connected to a control signal end with a CPU 41 described later via a pass line 36b.

The conference unit 6 has a CPU 41 for controlling the controller 36 and a keyboard 42 for inputting, for example, a request for searching for an image file to the server unit 5 and for inputting various information along with the image file. And the signal of the keyboard 42 and the CPU
A keyboard interface (hereinafter, referred to as a keyboard I / F) 43 for matching with the signal of 41, and a search monitor 44 for displaying information input by the keyboard 42
And a mouse 45 for giving an instruction to move the cursor coordinates on the screen of the search monitor 44 to an arbitrary position, and a mouse interface (hereinafter referred to as a mouse I / F) for matching a signal of the mouse 45 with a signal of the CPU 41. ) 46
A hard disk 47 on which an execution program of the CPU 41 and various data such as image data of a menu screen of the search monitor 44 are recorded; and a hard disk interface (hereinafter referred to as a hard disk I / F) for matching a signal of the hard disk 47 with a signal of the CPU 41. 4)
8, a working memory 49 used as various processing work areas of the CPU 41, and an image processing unit 5 including a memory for storing digital RGB video signals to be displayed on the search monitor 44.
0 and a printer 51 for printing information from the CPU 41
And a printer interface (hereinafter, referred to as a printer I / F) 5 for matching the printer 51 with the CPU 41.
2 is provided.

Further, the conference unit 6
A password storage unit 61 that stores a password out of information input from the keyboard 42 by the PU 41, a password monitoring unit 62 that determines the level of the password stored in the password storage unit 61, and a password monitoring unit 62 based on the determination result. And a control restriction unit 63 for restricting the control of the CPU 41.

The control signal end and the data signal end of the CPU 41 are connected to the hard disk I / F 4 by a bus line 41a.
8, mouse I / F 46, keyboard I / F 43, working memory 49, image processing unit 50, and password monitoring unit 61
Are connected to the control signal end and the data signal end. The CPU 41 uses the bus line 41a to control the hard disk I / F 48, the mouse I / F 46, and the keyboard I / F 46.
The F43, the printer I / F 52 and the working memory 49 are controlled.

The mouse I / F 46 detects a signal corresponding to the physical relative movement amount of the mouse 45 and outputs it to the work memory 49. The work memory 49 stores the movement amount described above.

The keyboard I / F 43 includes a keyboard 42
Is output to the working memory 49, and the working memory 49 stores the above-described character information and the like.

The hard disk I / F 48 reads a program executed by the CPU 41 and image data for the search monitor 44 such as a menu screen from the hard disk 47 and outputs the read data to the work memory 49. Image data and the like are stored.

The printer I / F 52 transmits information transmitted from the CPU 41 to the printer 51 and performs printing.

As described above, the CPU 41 loads the program stored in the hard disk 47 into the working memory 44 when the power is turned on, and operates according to the program.

As described above, the image file stored in the hard disk 23 of the server unit 5 is output to the hard disk driver 14 and temporarily stored in the memory 22 or directly stored in the LA 22 under the control of the controller 26.
Output to N controller 21. An image file is input from the LAN controller 21 to the LAN controller 31 of the conference unit 6 via the LAN cable 5a.

Then, in the conference unit 6,
The LAN controller 31 is controlled by the controller 36.
Outputs an image file to the image processing unit 32. The image file from the hard disk 23 of the server unit 5 stored in the image processing unit 32 is separated into digital observation image data and management information under the control of the controller 36, and the digital observation image data is stored in the image processing unit 32. The management information is sent to the CPU 41 via the bus line 36b.

The image processing section 32 stores the above-mentioned observation image data. As described above, the observation image data, which is a digital signal, stored in the image processing unit 32 is converted into an analog RGB video signal by inverse quantization of the D / A converter 34, and is output to the observation monitor 35. ing. The observation monitor 35 projects the input analog RGB video signal as described above. If the observation image data is compressed image data, it is output to the D / A converter 34 after being expanded by the expansion device 33.

The CPU 41 includes a cursor by the mouse 45, character information by the keyboard 42, the hard disk 4
The arithmetic processing is performed so that the image data for the search monitor 44 such as a menu screen from 7 and the management information are combined or displayed alone, and stored in the image processing unit 50 as image data.

The observation image data of the image file recorded on the hard disk 23 is, for example, a color observation image divided into 640 horizontal dots and 480 vertical dots, and the RGB color signal levels corresponding to these dots are set to, for example, It consists of a predetermined number of bytes quantized to 8 bits.

The diagnosis support processing using the endoscope diagnosis support apparatus having the above-described configuration uses the diagnosis support processing execution program recorded on the hard disk 47 and executes the CPU 4
1 is executed. The diagnosis support process is operated by input using the keyboard 42 and the mouse 45 in a multi-window environment displayed on the search monitor 44.

In the endoscope diagnosis support apparatus, various information regarding the patient / examination and the image transmitted from the server unit 5, the set position / shape information of the ROI, the accompanying finding information, and the ROI are calculated from the ROI. The hard disk 47 stores a database for storing the values of the feature amounts and the discrimination results obtained by applying the discrimination and classification processing. This database is accessed by the diagnosis support processing execution program.

Next, an outline of the conventional diagnosis support processing will be described with reference to FIG.

As shown in FIG. 2, the diagnosis support processing execution program 70 includes an image input / management block 71, a database management block 72, an ROI setting block 73, a feature amount calculation block 74, a discrimination classification block 75, and a report creation block 76. And an image processing block 77. Hereinafter, an outline of each block constituting the diagnosis support processing execution program 70 will be described.

The image input / management block 71 performs input, management, and search of an endoscope image used in the diagnosis: support processing.

The database management block 72 manages a database for storing and managing images used in the diagnosis support processing, ROIs to be set, calculated feature amounts, and the like.

The ROI setting block 73 sets an ROI for calculating a feature based on the mucosal surface structure on the endoscopic image.

The feature value calculating block 74 calculates a feature value to be used for the discriminant classification process in the discriminant classification block 75 and the statistical process in the report creation block 76 by applying the feature value calculating method.

The discrimination / classification block 75 performs a discrimination / classification process using the feature amount calculated in the feature amount calculation block 74.

The report creation block 76 displays a list of processing results obtained by the feature calculation block 74 and the discriminant analysis block 75, displays statistical processing results such as an average value of feature for each lesion type, and displays the results on a graph. Create a report by plotting.

The image processing block 77 applies image processing such as noise suppression processing, inverse γ correction processing, and structural component enhancement processing to the endoscope image used for the diagnosis support processing.

Then, the processing result image by the image processing block 77 is recorded on the hard disk 23 in the server unit 5 like the original image, and can be used as data to be subjected to diagnosis support processing.

Next, the operation of the conventional endoscope diagnosis support apparatus will be described with reference to FIGS.

In the following description, it is assumed that a series of processing is applied to the G image data which well reflects the mucosal surface structure among the R, G and B image data constituting the endoscope image. .

First, as shown in FIG. 3, in step S1, an endoscope image to be processed is input.
The endoscope image is sent from the server unit 5 to the LAN cable 5a.
Is input to the conference unit 6 via
The data stored in the database on the hard disk 47 can be obtained by being read by the database management block 72.

Next, a G image is obtained in step S2. Subsequently, in step S3, a noise suppression process for reducing random noise on the image is applied.

As noise suppression processing, filtering using a general averaging filter, Gaussian filter or median filter is applied to each pixel. At this time, the size of the filter (mask size) is determined by the degree of noise on the image. When the noise is small, the filter is set to, for example, about 3 × 3, and when the noise is large, a larger filter is used.

Next, in step S4, inverse gamma correction is applied to eliminate non-linear conversion for monitor display.

The processes in steps S3 and S4 are as follows:
This is executed in the image processing block 77 in FIG.

Subsequently, in the ROI setting block 73, a series of processes shown in steps S5 to S9 are applied. First, the counter i for counting the number of ROIs set in step S5 is set to 1.

Next, in step S6, the i-th ROI (i) is set on the G image data to be processed.
The setting of the ROI is performed, for example, by operating the mouse 45 and the keyboard 42 on the ROI setting window 100 shown in FIG.

In FIG. 4, an endoscope image display area 101 is shown.
The ROI is set using the mouse cursor 102 on the G image data displayed in (1). At this time, whether the ROI type is an arbitrary drawing shape or a fixed shape such as a rectangle is selected using the ROI type selection buttons 103a and 103b. Next, ROI information input / display area 1
At 04, information related to the ROI, such as the visual form of the lesion, the findings, the name of a diagnosis by a doctor, and a comment is input or selected, and the ROI setting button 105 completes the ROI setting. An ROI deletion button 106 for deleting a set ROI is displayed in the ROI setting window 100.
An end button 107 for ending the ROI setting and closing the ROI setting window 100 is provided.

Subsequently, the end of the ROI setting is determined in step S7. If the next ROI is to be set without terminating, the process proceeds to step S8. 4 corresponds to the case where a new ROI is set on the endoscope image display area 101. If it is finished, the process proceeds to step S9. In the example using FIG. 4, this corresponds to the case where the end button 107 is pressed, and the ROI setting window 100 is closed. At this time, the database management block 72 saves the position and shape of the set ROI and the assigned information in a database on the hard disk 47.

In step S8, the value of the counter i is increased by 1, and the process returns to step S6.

In step S9, the set ROI
, The process proceeds to step S10.

In step S10, the feature amount is calculated from the set ROI (1) to ROI (N) by applying the feature amount calculation method in the feature amount calculation block 74. As the feature amount, for example, JP-A-10-148
A statistic such as an average value of a feature using a Gabor filter disclosed in Japanese Patent No. 64 is calculated. Regarding the calculated feature amount, the database management block 72 uses the ROI
(1) The data is stored in a database on the hard disk 47 after being associated with each of the ROIs (N).

In step S11, the set R
The discriminant classification for OI (1) to ROI (N) is applied. More specifically, a feature vector obtained from a lesion group, such as a normal part, a cancer, an adenoma, or the like (these correspond to a class in the field of multivariate analysis), for which ROI setting and feature value calculation have already been performed, is represented by a feature vector ( These are called teacher data), and classification using a statistical or non-statistical classifier is applied in the discrimination classification block 75. The discriminator calculates a feature vector based on a feature amount obtained from data whose group (class) to which the classifier belongs to as teacher data, and applies the data to the data whose class is unknown to classify the data into one of the classes. It is classified into. It is also possible to calculate an index (called a distance from the class) indicating how similar the data is to each class. A class can be freely defined as long as it is a set of data having similar attributes (the findings correspond to this in the endoscope diagnosis support apparatus of the present embodiment). Pit pattern in colonic mucosa
(Types I, II, ...) and "regular and irregular groups".

As a statistical classifier, Fisher's discriminant function,
As a non-statistical classifier, a neural network is widely used as a representative one, and a type of distribution of a feature from a lesion to be classified (for example, a multivariate normal distribution or other distributions). ) And the performance of the discriminator itself and the like. Here, Fisher's discriminant function is used.

Finally, in step S12, the report creation block 76 displays diagnosis support information including the results of the discrimination and classification.

FIG. 5 shows a display example of the diagnosis support information. The report display window 110 is opened, and the ROI number of the endoscopic image for which the diagnosis support information is to be displayed, the classification result by the discriminator, and the diagnosis assumed from the classification result are displayed in the diagnosis support information display area 111. indicate. In this embodiment, the classification result (called a class) output from the discriminator is defined as a pit (pit) on the mucosal surface,
Diagnosis names diagnosed from the type (pit pattern) are also displayed. In the calculation of the classifier, the diagnosis name itself can be used as a class.

In the report display window 110, a feature value display button 112, a graph display button 1
13, print button 114, image / ROI display button 1
15 and an end button 116. The display of the value / average value of the feature amount calculated by the operation of the mouse 45, the display of the graph of the feature amount, the printing of the diagnosis support information, the image for the diagnosis support information display target and the ROI are provided. And the display of the diagnosis support information are ended.

Next, an outline of the endoscope diagnosis support apparatus according to the present embodiment will be described with reference to FIG.

As shown in FIG. 6, the diagnosis support processing execution program 80 in the present embodiment includes an image input / management block 81, a database management block 82, a temporary RO
I setting block 83, feature amount calculation block 84, ROI
It consists of a setting block 85, a discrimination and classification block 86, a report creation block 87, and an image processing block 88. Hereinafter, an outline of each block constituting the diagnosis support processing execution program 80 will be described.

The image input / management block 81 performs input, management and search of an endoscope image used in the diagnosis support processing.

The database management block 82 manages a database for storing and managing images used in the diagnosis support processing, ROIs to be set, calculated feature amounts, and the like.

[0091] The provisional ROI setting block 83 is used to obtain detailed diagnosis support information on the endoscope image.
Prior to the setting of I, an area to be roughly processed (hereinafter referred to as a temporary ROI) is set.

The characteristic amount calculating block 84 applies the ROI characteristic amount calculating method to the set temporary ROI, thereby setting the ROI in the ROI setting block 85 and discriminating using the characteristic amount calculated for each ROI. A feature value to be used for the discriminant classification processing in the classification block 86 and the statistical processing in the report creation block 87 is calculated.

The discrimination / classification block 86 performs a discrimination / classification process on the ROI set in the ROI setting block 85 using the feature calculated in the feature calculation block 84.

The report creation block 87 displays a list of the processing results obtained by the feature amount calculation block 84 and the discriminant analysis block 86, displays statistical processing results such as the average value of feature amounts for each lesion type, and displays the results on a graph. Create a report by plotting.

The image processing block 88 applies, for example, image processing such as noise suppression processing, inverse γ correction processing, and structural component enhancement processing to the endoscope image used for the diagnosis support processing.

The image processed by the image processing block 88 is recorded on the hard disk 23 in the server unit 5 as well as the original image, and can be used as data for diagnosis support processing.

Next, the operation of the endoscope diagnosis support apparatus according to the present embodiment will be described with reference to FIGS.

In the following description, it is assumed that a series of processing is applied to the G image data which well reflects the mucosal surface structure among the R, G and B image data constituting the endoscope image. .

As shown in FIG. 7, first, in step S
At 20, an endoscope image to be processed is input.
The endoscope image is sent from the server unit 5 to the LAN cable 5a.
Is input to the conference unit 6 via
The data stored in the database on the hard disk 47 can be obtained by being read by the database management block 82.

Next, in step S21, a G image is obtained. Subsequently, in steps S22 and S23, the noise suppression processing and the inverse γ correction processing shown in steps S3 and S4 in FIG. 3 are applied, respectively. The application of each process in steps S22 and S23 is executed in the image processing block 88 in FIG.

In the following step S24, the temporary ROI
The setting block 83 determines whether or not to set a temporary ROI. Step S2 when setting a temporary ROI
The process proceeds to step S5, and if not set, to step S26.
The provisional ROI is for omitting the processing for the area other than the lesioned part, and the setting by the operator is not always necessary in the present embodiment. If the provisional ROI is not set, the entire endoscope image is set as the ROI setting target in the subsequent processing. In this case, in the following description, the entire endoscope image is referred to as a temporary ROI.

In step S25, a temporary ROI is set. The provisional ROI is for reducing the amount of calculation in the processing after step S26, and for avoiding the display of the diagnosis support information for the area that is not the affected part. That is, an ROI for applying the original diagnosis support processing is set in the set temporary ROI.

For setting the temporary ROI, for example, in the temporary ROI setting window 200 shown in FIG.
5 and the keyboard 42 are operated.

In FIG. 8, an endoscope image display area 201
A temporary ROI is set using the mouse cursor 202 on the G image data displayed in (1). Here, it is assumed that a figure shown by a broken line in the endoscope image display area 201 is set as a temporary ROI. Whether the type of the temporary ROI is set to an arbitrary drawing shape or a fixed shape such as a rectangle is selected using the temporary ROI type selection buttons 203a and 203b. Subsequently, in the provisional ROI information input / display area 204, information associated with the provisional ROI such as the visual form of the lesion, findings, a doctor's diagnosis name, and comments is input or selected, and the provisional ROI setting button 205 completes the ROI setting.
Further, the provisional ROI setting window 200 is provided with a provisional ROI deletion button 206 for deleting the provisional ROI, and an end button 207 for ending the provisional ROI setting and closing the provisional ROI setting window 200.

In the following step S 26, the feature amount calculation is performed for each surface element in the set temporary ROI by applying the feature amount calculation method in the feature amount calculation block 84. In the present embodiment, a Gabor feature amount h (X, Y, θ _k , λ _m ) using a Gabor filter disclosed in Japanese Patent Application Laid-Open No. H10-14864 is used as the feature amount. Here, X and Y are the coordinates of the pixel in the image, and θ _k and λ _m are parameters that define the direction and wavelength of the Gabor filter, respectively. In this embodiment, a Gabor filter having four directions and two frequencies is used.
It is assumed that 1 ≦ k ≦ 4 and 1 ≦ m ≦ 2. These G
Eight feature values are calculated for each surface element by applying the abor filter. These are calculated for the i-th pixel by Ci =
{H1i, h2i, h3i,. . . , H8i} in vector notation (feature vector). i is an integer of 1 or more indicating the number of a pixel in the temporary ROI. Also, h1i to h8i indicate eight Gabor feature amounts calculated for the pixel i.

Next, in step S27, step S
The temporary ROI is divided into N (N is an integer equal to or greater than 1) ROIs by a region dividing process using the feature amount of each surface element calculated in step S26.

The area dividing process in step S27 will be described with reference to FIG. The region division processing is to obtain a divided ROI by integrating the pixels in the temporary ROI by utilizing similar values obtained from pixels in the region showing the same finding. There are various methods for region division processing using feature amounts. In the present embodiment, a K-means algorithm, which is a typical method of region division (generally referred to as clustering) using feature amounts, is used. I do.

In the present embodiment, the feature value Ci of each pixel calculated in step S26 = {h1i, h2i, h3i,. . . , H8i} are used in a series of processes after step S131 shown in FIG.

In step S131, an initial area is set for the temporary ROI. FIG. 10 shows an example of setting the initial area. Here, it is set so that a rectangle having a maximum size of L × L is mechanically obtained. However, the area need not necessarily be rectangular, and a region including the boundary of the temporary ROI (solid line in FIG. 10) is used near the boundary of the temporary ROI. The value of L is determined based on the processing target. For example, L = 5 for an observation image of a large intestine pit pattern.

In the following step S132, the average value of the feature amount Cmeanj = {h1meanj, h2meanj for each region
, ・ ‥, h 8meanj}. Here, j is an integer of 1 or more indicating the number of the set area.

In step S133, C of each pixel is
i is compared with Cmean of each area by a distance shown in the following equation.

[0112]

(Equation 1) Here, α1 to α8 indicate weighting coefficients for each feature amount. Here, α1 = α2 =. . . = Α8 = 1. Based on Dij obtained for each region,
The area is reconstructed by integrating the picture i into the area giving the minimum Dij.

In step S134, step S1
Similarly to 32, the average value of the feature amount for each region after the reconstruction is calculated.

In step S135, convergence is confirmed by a change in the average value of the feature values for each reconstructed region. If the amount of change is equal to or larger than the threshold, the process returns to step S133, and the region is reconstructed again. If it is less than the threshold value, it is considered that the reconstruction of the area has converged, and the process proceeds to step S136 to complete the area division processing.

Here, the threshold value for the convergence determination is a value experimentally determined based on the target endoscope image and the characteristic amount. For example, the threshold value of the difference between Cmeanj before and after the reconstruction is 1 0. Set to 0.

FIG. 11 shows an ROI obtained by applying the area dividing process.
Here is an example of the settings. Temporary ROI2 shown by a broken line in FIG.
10 is divided into four ROIs. Each ROI is
Each of the calculated feature amounts is an area determined to have a different tendency from the adjacent ROI. Set N ROIs (N is an integer of 1 or more)
For each of the ROIs (1) to ROI (N), the ROI is calculated by the feature amount calculation block 84 in step S28 in FIG.
The feature amount of each is calculated.

The difference between the feature amounts calculated in steps S26 and S28 will be described. The feature amount in step S26 is calculated for each surface element, and a similar value is obtained by using similar values obtained from pixels in the region showing the same finding.
It is used to integrate the pixels in the OI to obtain ROI (i) (1 ≦ i ≦ N). On the other hand, step S2
The feature amount in 8 is calculated for each obtained ROI (i), and is, for example, the average value or variance of the feature amount of each surface element in each ROI, and is used in the subsequent discrimination and classification processing. The feature amount calculation method applied in each step may be the same or different, and is selected according to the type of the lesion to be diagnosed.

In the present embodiment, step S26
Gabor of pixel included in each ROI calculated in
Average value μ (1) to μ of feature value using filter
(N) and standard deviations σ (1) to σ (N) are defined as ROI (1).
Or ROI (N).

Subsequently, a series of processes shown in steps S29 to S32 are applied in the discrimination / classification block 86 using the feature amount calculated for each ROI. First, step S
At 29, the value of the counter i is set to 1.

In step S30, ROI (i)
On the other hand, a discriminant classification result is obtained by applying the discriminator in the same manner as shown in step S11 in FIG.

In step S31, it is determined whether or not processing has been completed for all N ROIs. If not, the value of i is increased by 1 in step S32, and the flow returns to step S30. Step S if completed
Go to 33.

In step S33, the report creation block 87 displays the diagnosis support information including the result of the classification.

FIG. 12 shows a display example of the diagnosis support information.
The report display window 220 is opened, the ROI number of the endoscopic image for which the diagnosis support information is displayed, the classification result by the classifier, the diagnosis name assumed from the classification result, and the area of each ROI in the temporary ROI. The ratio is displayed in the diagnosis support information display area 221.

In the report display window 220, a feature value display button 222, a graph display button 223, a print button 224, an image / ROI display button 2
25 and an end button 226. The display of the value / average value of the feature amount calculated by the operation of the mouse 45, the display of the graph of the feature amount, the printing of the diagnosis support information, the image to be displayed and the ROI of the diagnosis support information are provided. And the display of the diagnosis support information are ended.

As described above, according to the endoscopic diagnosis support apparatus of the present embodiment, it is possible to set the ROI without fail for a lesion for which diagnosis support information is desired to be obtained. In addition, it is possible to present what findings and diseases are present at what ratios for each ROI.

In the present embodiment, the calculation of the characteristic amount from the G image has been described. However, an R or B image can be used. It is also possible to use a luminance image obtained by conversion using each color signal. Further, it is also possible to calculate a feature amount by combining a plurality of images such as R / (R + G + B) or R / G.

Further, the feature amount calculating method used for the area division processing for setting the ROI for the temporary ROI may be different from the feature amount calculating method used for the identification and classification for the set ROI. In that case, for example, R / (R + G + B) is used for the area division processing, and Gabo is used for the identification and classification processing.
A feature based on an r filter is used.

Further, the provisional RO of the characteristic amount is used as the diagnosis support information.
Information such as the average value, the standard deviation, and the classification result of the whole I may be displayed.

Second Embodiment: In a second embodiment of the present invention, ROIs are set for all lesions for which diagnosis support information is desired to be obtained, and what findings and diseases are determined for each ROI. Endoscopic diagnosis support apparatus that can present the ratio of the presence of the target, and can present more comprehensive diagnosis support information for the lesion based on the diagnosis support information for each ROI. It is about.

The configuration of the endoscope diagnosis support apparatus according to the present embodiment and the configuration blocks of the diagnosis support processing execution program are the same as those of the endoscope diagnosis support apparatus described in the first embodiment. Therefore, different points will be described below.

FIGS. 13 to 16 relate to the second embodiment of the present invention. FIG. 13 is a block diagram for explaining the configuration of a diagnosis support processing execution program according to this embodiment. 15 is a flowchart for explaining the operation of the program, FIG. 15 is a flowchart for explaining the derivation of the diagnosis support information, and FIG. 16 is a diagram for explaining the display of the diagnosis support information in the endoscope diagnosis support device of the present embodiment. It is a report display window explanatory view.

As shown in FIG. 13, the diagnostic support processing execution program 230 includes an image input / management block 231,
It comprises a database management block 232, a provisional ROI setting block 233, a feature amount calculation block 234, an ROI setting block 235, a discrimination and classification block 236, a comprehensive diagnosis block 237, a report creation block 238, and an image processing block 239. Among them, the blocks other than the comprehensive diagnosis block 237 and the report creation block 238 perform the same processing as each block in the diagnosis support processing execution program 80 described in the first embodiment.

The comprehensive diagnosis block 237 is for obtaining comprehensive diagnosis support information for the entire lesioned part in which the temporary ROI has been set, based on the results of the determination and classification of each ROI obtained by the determination and classification block 236.

The report creation block 238 in this embodiment displays the comprehensive diagnosis support information obtained by the comprehensive diagnosis block 237 in addition to the display of the diagnosis support information shown in the first embodiment. .

Next, the operation of the endoscope diagnosis support apparatus according to the present embodiment will be described with reference to FIG. In the present embodiment, steps S20 to S32 of the operation described with reference to FIG. 7 in the first embodiment are the same.

After the discrimination / classification processing is applied to all ROIs in step S31, the process proceeds to the acquisition of comprehensive diagnostic information in step S41. Here, the comprehensive diagnostic information is diagnostic support information for the entire temporary ROI obtained as follows.

For example, in the diagnosis based on the morphology of the large intestine pit pattern, what type of pits are present and at what ratio is an important diagnostic index. Based on the discrimination classification information obtained from each ROI, FIG.
The following judgment is made.

In FIG. 15, each pi is obtained from step S51.
Whether or not the t pattern exists and its ratio is determined in order from the disease with the highest malignancy. Note that the numerical value shown as the probability in FIG. 15 is a numerical value indicating how often the disease to be determined appears in the classification result. For example, steps S51, S52 and S
The case of passing through 53 indicates that "the probability that a lesion in which a V-type pit pattern is present at a rate of 20% or more is early cancer invading under the mucosa is 60% or more".

In step S42 of FIG. 14, the report creation block 87 displays diagnosis support information including comprehensive diagnosis information. FIG. 16 shows a display example of the diagnosis support information. In the report display window 240, comprehensive diagnostic information is displayed in the comprehensive diagnostic information display area 241 in addition to the items described with reference to FIG. 12 in the first embodiment.

As described above, according to the endoscope diagnosis support apparatus of the present embodiment, the ROI is set for all the lesions for which the diagnosis support information is desired to be obtained, and each RI is set.
It is possible to present what findings and diseases exist in each OI and at what proportion,
It is possible to present more comprehensive diagnosis support information for a lesion based on the diagnosis support information for OI.

Third Embodiment: A third embodiment of the present invention is another form of the endoscope diagnosis support apparatus shown as the second embodiment, and it is desired to obtain diagnosis support information. ROIs can be set for all affected lesions, and what findings and proportions of diseases are present for each ROI can be presented, and lesions can be presented based on diagnosis support information for each ROI. The present invention relates to an endoscope diagnosis support device capable of presenting more comprehensive diagnosis support information to a patient.

The configuration of the endoscope diagnosis support apparatus and the configuration block of the diagnosis support processing execution program according to the present embodiment are the same as those of the endoscope diagnosis support apparatus described in the second embodiment. Therefore, different points will be described below.

FIG. 17 is an explanatory view of a report display window for explaining display of diagnosis support information in the endoscope diagnosis support apparatus according to the third embodiment of the present invention.

In the second embodiment, the comprehensive diagnostic information of the entire lesion is obtained by referring to the judgment criterion illustrated in FIG. 15, but in the present embodiment, the following diagnostics are performed in step S41 of FIG. By performing the processing, comprehensive diagnostic information is obtained.

Steps S20 to S31 in FIG.
By the series of processing shown in (1), the discrimination classification result for each of the N ROIs and the area ratio occupied in the temporary ROI are calculated. In the present embodiment, the discrimination / classification processing is applied again using the area ratio of the obtained discrimination / classification result as a feature amount. For example, type I,
When seven types of type II, IIIs, IIIL, IV, Va, and VN are assumed, a seven-dimensional feature vector can be formed by the value of the area ratio of each pit pattern. Each pitpattern
Are 0.2, 0.0, 0.3, 0.1, 0.0,
Assuming that 0.2 and 0.2, the feature vector X is X = {0.2, 0.0, 0.3, 0.1, 0.0, 0.2, 0.2}. Therefore, by applying to the classifier the diagnosis name of a disease such as cancer or gland type in advance and applying it to the classifier created using the teacher data with the area ratio of each pit pattern in the lesion as a feature vector, the lesion can be identified. It is possible to indicate which of the diseases is classified.

As described above, according to the endoscope diagnosis support apparatus of the present embodiment, the ROI is set for all the lesions for which the diagnosis support information is desired to be obtained, and each RI is set.
It is possible to present what findings and diseases exist in each OI and at what proportion,
It is possible to present more comprehensive diagnosis support information for a lesion based on the diagnosis support information for OI.

Fourth Embodiment: In a fourth embodiment of the present invention, display of diagnosis support information is performed on the endoscope diagnosis support apparatus shown as the first, second, and third embodiments. The present invention relates to an endoscope diagnosis support apparatus capable of presenting only important information by not displaying information that is not important for diagnosis.

The configuration of the endoscope diagnosis support apparatus according to the present embodiment and the configuration blocks of the diagnosis support processing execution program are the same as those of the endoscope diagnosis support apparatus described in the second embodiment. Therefore, different points will be described below.

For example, in a diagnosis based on the morphology of the large intestine pit pattern, the type I pit pattern is a finding of a normal mucosa and may not be important information as compared with other pit patterns. Therefore, in the display of the diagnosis support information in step S42 shown in FIG. 14, the display of the diagnosis support information for the ROI whose discrimination / classification processing result is determined to be the I-type pit pattern as shown in FIG. 17 is omitted.

At this time, the discrimination classification result for omitting the information display is registered in the database stored in the hard disk 47 or a dedicated setting file, and is referred to when the report is created by the report creation block 238.

As described above, according to the endoscope diagnosis support apparatus of the present embodiment, important diagnosis support information can be selectively selected even when an extremely large number of ROIs are set for a temporary ROI, for example. Can be presented.

Fifth Embodiment: The fifth embodiment of the present invention relates to the endoscope diagnosis support apparatus shown as the first to fourth embodiments, and uses a normal endoscope image (hereinafter referred to as “endoscope image”). , A normal image) or an endoscopic image (hereinafter, referred to as a stained image) on which the medicine is scattered, and appropriately select an image to be processed in the feature amount calculation from each of the RGB images. The present invention relates to an endoscope diagnosis support apparatus that can always provide good diagnosis support information.

The configuration of the endoscope diagnosis support apparatus according to the present embodiment is the same as that of the endoscope diagnosis support apparatus described in the first to fourth embodiments. Therefore, different points will be described below.

FIGS. 18 to 20 relate to the fifth embodiment of the present invention. FIG. 18 is a block diagram for explaining the configuration of the diagnosis support processing execution program, and FIG. 19 shows the operation of the diagnosis support processing execution program. FIG. 20 is a flow chart for explaining, and FIG. 20 is an explanatory diagram for explaining the judgment processing of the normal image and the stained image.

As shown in FIG. 18, the diagnostic support processing execution program 250 includes an image input / management block 251, a database management block 252, an image type determination block 253, a temporary ROI setting block 254, a feature amount calculation block 255, and an ROI setting. It comprises a block 256, a discrimination and classification block 257, a report creation block 258, and an image processing block 239. Among them, the blocks other than the image type determination block 253 perform the same processing as each block in the diagnosis support processing execution program 80 described in the first embodiment.

In the image type determination block 253, the following series of processing is applied to the input endoscope image. Here, a description will be given of a case where three kinds of drugs of indigo carmine, methylene blue and crystal violet are selectively used.

In the present embodiment, steps S20 and S21, which are the operations of the diagnosis support processing execution program 80 shown in FIG.
The processing is changed to steps S100 to S107 in FIG.

In step S100, an endoscope image to be processed is input. Next, an image type determination process described later is performed in step S101.

In step S102, it is determined whether or not the image type determination result is a normal image. If Yes, the process proceeds to step S106. If No, the process proceeds to step S10.
Proceed to 3.

In step S103, it is determined whether the image type determination result is a stained image by crystal violet. If the determination result is Yes, the process proceeds to step S106, and if No, the process proceeds to step S104.

In step S104, it is determined whether the image type determination result is a methylene blue stained image. If the determination result is Yes, step S107 is performed.
If No, go to step S105.

In step S105, it is determined whether or not the image type determination result is an indigo carmine stained image. If the determination result is Yes, step S10 is performed.
If No, go to step S106. If No is determined in step S105, the image may be a stained image using another drug. In this embodiment, the G image is selected at this time.

In step S106, it is assumed that the G image of the RGB images is applied to the processing after step S21 shown in FIG. 7, and the process proceeds to step S22.

In step S107, it is assumed that the R image in the RGB image is applied to the processing after step S21 shown in FIG. 7, and the process proceeds to step S22.

Next, the processing content of the image type determination in step S101 will be described. In the present embodiment, the image type is determined according to the overall color tone of the endoscope image.

Let the pixel values of the RGB image in the input endoscope image be (ri, gi, bi). Here, i is a continuous number assigned to each surface element and is an integer of 1 or more. For example, when the size of the image is X × Y and all the pixels are used for determination, 1 ≦ i ≦ X × Y. When M pixels are used by sampling or the like, 1 ≦ i ≦ M. Hereinafter, it is assumed that M pixels are used.

From each surface element, Si = tan ⁻¹ (gi / r
i), Ti = tan ^-1 (bi / ri) is obtained. ri, gi
And bi take a value of 0 or more (for example, 0 to 255 in an 8-bit gradation), so that the values of Si and Ti are 0 ≦ S
i, Ti are real numbers in the range of <90. In practice, an approximate value obtained by dividing the range into 90 is used. For example, it is possible to discretize each of 0, 1, 2,..., 89 by rounding off at the first decimal place.

Si and Ti are the values of each surface element (ri, gi, b
The value is defined by the ratio of i), and the distribution of the obtained values differs between the normal image and various stained images. Specifically, the normal image having a large value of ri is Si and Ti.
Indigo carmine and methylene blue, which are blue-based drugs, have a smaller value of ri than that of a normal image, and both Si and Ti are large. Also, methylene blue has a lower gi than indigo carmine, so that Ti> Si. On the other hand, crystal violet, which is a purple-colored drug, has large values of ri and bi with respect to the value of gi, so that Si is small and Ti is large. Figure 20 shows the differences between these distributions. In FIG.
(1) shows a normal image, (2) shows a stained image with indigo carmine, (3) shows a stained image with methylene blue, and (4) shows a region where a large number of pixels are distributed in a crystal violet stained image. With this property, for example, when 60% or more of the M pixels are included, it can be determined that the image type corresponds to the corresponding area. In addition, when it is difficult to determine any of the image types, such as the distribution is very rare, the image is determined to be unknown.

As described above, according to the endoscope diagnosis support apparatus of the present embodiment, the endoscope diagnosis support apparatus uses either a normal endoscope image or an endoscope image on which a medicine is scattered. It is possible to determine whether or not there is, and to appropriately select an image to be processed in the feature amount calculation from each of the RGB images, thereby always providing good diagnosis support information.

Further, it is of course possible for the operator to manually perform the image type determination.

When the determination is impossible, the number of pixels used for the determination may be increased or the position may be changed, and the determination may be performed again.

Sixth Embodiment A sixth embodiment of the present invention relates to the endoscope diagnosis support apparatus shown as the first to fifth embodiments, and is either a normal image or a stained image. Endoscope diagnosis support apparatus that can provide good diagnosis support information by enabling each image to be used in a mixed manner by inverting the gradation according to the result. It is about.

The configuration of the endoscope diagnosis support apparatus according to the present embodiment and the configuration blocks of the diagnosis support processing execution program are the same as those of the endoscope diagnosis support apparatus described in the fifth embodiment. Therefore, different points will be described below.

FIG. 21 is a flowchart for explaining the operation of the diagnosis support processing execution program according to the sixth embodiment of the present invention.

In the stained image, as shown in FIG. 23, the appearance of the mucosal surface structure on the image is reversed due to the difference in the properties of the drugs used. For example, in observing the morphology of the pits, in the stained image using indigo carmine, the pits have a lower pixel value than the surrounding area, so that the normal image and methylene blue, crystal violet, etc. are absorbed by the living body. Is different from a dye having the property of obtaining an effect. Therefore, for example, when applying a feature amount calculation method involving binarization, the processing results will be different, and it is necessary to invert the gradation.

FIG. 21 is an explanatory diagram for explaining the operation of the diagnosis support processing execution program 250 according to the present embodiment. In the present embodiment, steps S20 and S21, which are operations of the diagnosis support processing execution program 80 shown in the first embodiment described with reference to FIG.
The processing is changed to steps S110 to S120.

In FIG. 21, the contents of each processing shown in steps S110 to S115 are the same as those in steps S100 to S105 in FIG. 19 in the fifth embodiment, respectively.

In step S112, it is determined whether or not the image type determination result is a normal image. If Yes, the process proceeds to step S116. If No, the process proceeds to step S113.
Proceed to.

In step S113, it is determined whether or not the image type determination result is a stained image using crystal violet. If the determination result is Yes, step S1 is performed.
If No, go to Step S114.

In step S114, it is determined whether or not the image type determination result is a methylene blue stained image. If the determination result is Yes, step S117 is performed.
If No, the process proceeds to step S115.

In step S115, it is determined whether or not the image type determination result is an indigo carmine stained image. If the determination result is Yes, step S11 is performed.
If No, go to Step S118. If No is determined in step S115, there is a possibility that the image is a stained image using another drug. In this embodiment, the G image is selected at this time.

In steps S116 and S118, the G image is selected as an image to be processed thereafter, and the flow advances to step S21.

In step S117, the R image is selected as an image to be processed thereafter, and the flow advances to step S21.

In step S119, the R image is selected as an image to be processed thereafter, and the flow advances to step S120.

In step S120, a tone reversal process is applied to enable mixed use of indigo carmine and other types of images. In the gradation inversion process, the image processing block 259 converts a pixel value using the following equation.

R′j = 255−rj where rj is the value of pixel j in the R image (1 ≦ j ≦ X
× Y, X and Y are the vertical and horizontal sizes of the image), and are 8-bit gradation numbers, and assume a range of 0 ≦ rj ≦ 255. r'j is the value of the pixel after gradation inversion and is used for the subsequent processing. After applying the gradation inversion process to the R image in step S120, the process proceeds to step S21.

As described above, according to the endoscope diagnosis support apparatus of the present embodiment, it is determined whether the image is a normal image or a stained image, and the gradation is inverted according to the result. Accordingly, it is possible to use each image in a mixed manner, and it is possible to provide good diagnosis support information.

Seventh Embodiment: The seventh embodiment of the present invention relates to the endoscopic diagnosis assisting apparatus shown as the first to fifth embodiments, and relates to a stained image using indigo carmine. By judging which of the other endoscope images it is, and by inverting the gradation according to the result, it is possible to use each image in a mixed manner, thereby providing good diagnosis support information. The present invention relates to an endoscope diagnosis support device that can be provided.

In the sixth embodiment of the present invention, it is determined whether the input endoscope image is a normal image or three types of stained images, and the difference in the properties of the used drugs is considered. Above, the endoscope diagnosis support apparatus that selects an R or G image and further applies a gradation inversion process to a stained image using indigo carmine has been described.

On the other hand, as described above, indigo carmine contains a large amount of structural components on the mucosal surface in the R image due to its bluish green color tone, but also in the G image, it is not sufficient to calculate the characteristic amount due to the properties of the original endoscopic image. In such a case, the G image can be commonly used.

In the present embodiment, the G image is commonly used in applying a feature amount calculation method from various endoscope images, and the gradation conversion process is applied only to a stained image using indigo carmine. The following describes an endoscope diagnosis support apparatus that simplifies the processing in the sixth embodiment, saves memory resources, and speeds up the processing.

The configuration of the endoscope diagnosis support apparatus and the configuration blocks of the diagnosis support processing execution program according to the present embodiment are the same as those of the endoscope diagnosis support apparatus described in the fifth embodiment. Therefore, different points will be described below.

FIG. 22 is a flowchart for explaining the operation of the diagnosis support processing execution program according to the seventh embodiment of the present invention.

In the present embodiment, steps S20 and S21, which are the operations of the diagnosis support processing execution program 80 shown in FIG.
The processing is changed to the processing shown in steps S140 to S144 in FIG.

In FIG. 21, steps S140 and S140
The content of each process indicated by 141 is the same as that of steps S100 and S101 in FIG. 19 in the fifth embodiment, respectively.

In step S142, a G image is obtained from the input endoscope image.

In step S143, it is determined whether or not the image type determination result is an indigo carmine stained image. If the determination result is Yes, step S14 is performed.
If No, go to step S121.

In step S144, a gradation inversion process for enabling mixed use of indigo carmine and other types of images is applied. In the gradation inversion processing, the image processing block 259 converts the pixel value using the following equation.

G'j = 255-gj where gj is the value of pixel j in the G image (1.ltoreq.j.ltoreq.X
× Y, X and Y are the vertical and horizontal sizes of the image), and are 8-bit gradation numbers, and assume a range of 0 ≦ gj ≦ 255. g ′ j is the value of the pixel after the grayscale inversion and is used for the subsequent processing. After applying the gradation inversion process to the G image, the process proceeds to step S21.

As described above, according to the endoscope diagnosis support apparatus of the present embodiment, it is determined whether the image is a stained image using indigo carmine or another endoscope image. By inverting the gradation in accordance with the above, it is possible to use each image in a mixed manner, thereby providing good diagnosis support information.

Eighth Embodiment: The eighth embodiment of the present invention relates to the endoscope diagnosis support apparatus shown as the first to seventh embodiments, and uses a spatial frequency analysis method in calculating a feature amount. The present invention relates to an endoscope diagnosis support apparatus capable of obtaining highly accurate diagnosis support information by using all information of a spatial frequency component of an endoscope image.

The configuration of the endoscope diagnosis support apparatus according to the present embodiment is the same as that of the endoscope diagnosis support apparatus described in the first embodiment. Further, the diagnosis support processing execution program in the present embodiment is the same as the diagnosis support processing execution program 80 shown in FIG. 6, and achieves its object by changing the feature calculation method applied in the feature calculation block 84. Is what you do.

The spatial frequency analysis technique is to extract frequency components by applying filtering to an image and to convert them into feature quantities. For example, a power spectrum method described below is known as the most general technique.

The image t (x, y) and the filter f (x,
The filtering result g (x, y) by y) is

G (u, v) = T (u, v) · F (u, v) (2) Here, G (u, v), T (u, v)
And F (u, v) are g (x, y), t (x,
y) and f (x, y). The filter f (x, y) has a low-frequency, high-frequency or band-pass type frequency characteristic, and has an effect of extracting a specific frequency component in an image. Equation (2) is expressed in real space by the convolution operation of the image and the filter.

G (x, y) = t (x, y) * f (x, y) (3), which is generally executed by digital filtering using an FIR filter or the like.

G (u, v) is a complex number, and its real and imaginary terms are represented by Re (G (u, v)) and Im, respectively.
(G (u, v))

G (u, v) = Re (G (u, v)) + jIm (G (u, v)) (4) Here, j indicates an imaginary unit. Further, from equation (4), G (u, v) is calculated using the amplitude term A (u, v) and the phase term ψ (u, v).

G (u, v) = A (u, v) exp (jψ (u, v)) (5)

(Equation 6)

７ (u, v) = tan ⁻¹ {Im (G (u, v)) / Re (G (u, v)}} (7)

In the power spectrum method, the amplitude term A (u, v) shown in the equation (6) is obtained from n (n is an integer of 1 or more) filters f (x, y) to which the amplitude term A (u, v) is applied. To use.

A Gabor feature which has been widely used in the field of image analysis in recent years is that a frequency characteristic of a filter f (x, y) is determined by taking into account a characteristic of a human visual system.
This is set by a function, and the amplitude term A (u, v) is used as a feature amount essentially as in the power spectrum method.

As shown in the equation (5), the image has an amplitude term A (u, v) and a phase term と し て
Although (u, v) is included, the latter loses information because it is not used as a feature value. In the present embodiment, the accuracy of the diagnosis support information is improved by calculating the feature based on the phase term ψ (u, v) in addition to the feature based on the amplitude term A (u, v).

In the present embodiment, the endoscope image
K × M Gabor filters for extracting frequency components
Shall be used. K and M are Gabor filters
This is the number of parameters that will be described later. Gaborf
Filters travel in one direction on two-dimensional Gaussian surfaces and two-dimensional planes.
With a plane wave
Standard value difference σ_xAnd σ_y, The traveling direction θ of the plane wave_kAnd waves
Long λ_mIs determined by Where k and m are
The traveling direction and wavelength parameters that define the Gabor filter
A meter, where 0 ≦ k <K and 0 ≦ m <M
I do. Standard value difference σ _xAnd σ_yIs the wavelength λ_mClosely related to
The wavelength λ_m, And σ_x(Λ
_m) And σ_y(Λ_m). Gabor filter f
Is a two-dimensional file consisting of the real part Re (g) and the imaginary part Im (g).
Iruta, respectively,

(Equation 8)

(Equation 9) Is defined by The processing result g of the convolution operation between the image t of size N × N and the Gabor filter f is the pixel t
(X, Y)

(Equation 10) Given by

Using g in equation (10), the value hk, m (X, Y) of the Gabor feature is

H _{k, m} (X, Y) = | g _{k, m} (X, Y) | (11) Here, | · | is the absolute value of the complex number α + jβ (α
² + β ² ) represents ^1/2 .

[0211] In the present embodiment, reference 4 [Rotation-Invariant T] is used as a feature using amplitude and phase information.
exture Classification Using a Complete Space-Frequ
encyModel, GMHaley and BSManjunath, IEEE TRANS.
ON IMAGE PROCESSING, Vol. 8, No. 2, FEB. 1999].

[0212]

(Equation 12)

(Equation 13)

[Equation 14]

(Equation 15)

(Equation 16)

[Equation 17] Here, nx and ny are the sizes ISX × ISY, respectively.
0 ≦ nx <ISX, 0
≤ny <ISY. Also, it s is a parameter corresponding to the wavelength lambda _m the Gabor filter, is 1 ≦ s ≦ M. R is a parameter corresponding to the direction θ _{k of the} Gabor filter, and 0 ≦ r <K. Here, it is assumed that R = K in accordance with the description in Reference 4. Also, ar
g [•] indicates tan ^-1 {Im (•) / Re (•)}.

Also, a _{s, r} (nx, ny) and φ _{s, r} (n
x, ny) and u _{s, r} (nx, ny) are obtained as follows.

A _{s, r} (nx, ny) = | g _{s, r} (nx, ny) | (18)

[Equation 19]

(Equation 20) Here, ∇ _x () and ∇ _y () are gradient estimation functions, and θ _r is G
The direction of the abor filter. Also,

Equation 21] _{^{θ∇ = tan -1 {▽ y (}} ψ s, r (nx, ny)) / ▽ x (ψ s, r (nx, ny)} is obtained from (21). Also, arg [· ] Is atan (Im
(•) / Re (•)).

The features shown in equations (# 11) to (# 16) are calculated for each surface element, and can be applied to step S26 in FIG. 7 shown in the first embodiment. .

Further, it is possible to calculate the following characteristic amounts for each region from the characteristic amounts for each pixel obtained by the equations (12) to (17).

[0216]

(Equation 22) Here, fA, fF, fY, fDA , fDF and fDY each _{fA s, p, fF s,} q, fY s, q, fDA s, q, fDF s, q and fDY
Vector notation based on p or _q for _{s, q} .
In addition, E 期待 represents an expected value (average value) of going to each plane element in the region to be calculated (ROI), and * represents a complex conjugate.

It is also possible to calculate feature values based on the variance and covariance of each feature value. For example, the expression (1
The variance and covariance using fA _{s, p} and fF _{s, q} shown in 2) and (13) are calculated as a variance-covariance matrix Σ. For example, when s = 1 and R = 8, fA _{s, p} and fF _{s, q} are each calculated three by three for each surface element, and set to j-th pixel (coordinate (nx, ny)) in the ROI. On the other hand, v1j = fA
_0,0 (nx, ny), v2j = fA _0,1 (nx, ny), v3
j = fA _0,2 (nx, ny), v4j = fF _0,0 (nx, n
_{y), v5j = fF 0,1 (} nx, ny), v6j = fF 0,2 (n
x, ny), the covariance matrix Σ is

(Equation 23) Is defined by In equation (23), σ _st is

(Equation 24) Required by Here, N is the number of pixels in the ROI, μ
s and μt indicate the average values in the ROI of vs and vt, respectively. In the obtained covariance matrix Σ, _replace each σ _st with the ROI
Is used as the feature value of. Similarly, the variance-covariance matrix can be obtained including the feature amounts shown in Expressions (14) to (17).

Further, a correlation matrix can be obtained from the covariance matrix Σ, and a correlation coefficient can be used as a feature value.

In this embodiment, it is possible to apply to step S27 in FIG. 7 shown in the first embodiment, and it is possible to apply each ROI after application of the area division processing.
From (i), the variance and covariance (or correlation coefficient) of FCA (i) to FDMY (i ) are obtained and used for the discriminant classification processing in step S29.

As described above, according to the endoscope diagnosis support apparatus of the present embodiment, when the spatial frequency analysis method is applied to the calculation of the feature amount, all information having the spatial frequency components of the endoscope image is obtained. , It is possible to obtain highly accurate diagnosis support information.

[Supplementary notes] (Supplementary note 1) Frequency component extracting means for extracting a predetermined frequency component from an image pickup signal of an object, and detecting phase information of the frequency component extracted by the frequency component extracting means. A diagnosis support apparatus comprising: phase information detection means; and feature value calculation means for calculating a feature value of the subject based on the phase information detected by the phase information detection means.

(Additional Item 2) An endoscope image input means for inputting an endoscope image composed of a plurality of color signals, and a first endoscope image input to the endoscope image input means. First area setting means for setting an area; feature quantity calculating means for calculating a feature quantity based on at least one color signal of the endoscope image; and a feature quantity calculating means based on the feature quantity calculated by the feature quantity calculating means. A second area setting means for setting a second area in the first area set by the first area setting means; and the characteristic for at least one area set by the second area setting means. Endoscope diagnosis support, comprising: a discriminating / classifying means for applying a discriminating / classifying process based on the feature amount calculated by the quantity calculating means; apparatus.

(Additional Item 3) The endoscope diagnosis support apparatus according to Additional Item 2, wherein the display means displays the feature amount calculated by the feature amount calculating means.

(Additional Item 4) The endoscope according to additional item 2 or 3, further comprising an area calculating means for calculating an area of the second area set by the second area setting means. Diagnosis support device.

(Supplementary note 5) The supplementary note, wherein the second region is provided with an area ratio calculating means for calculating an area ratio, and the display means displays the area ratio calculated by the area ratio calculating means. Item 5. The endoscope diagnosis support device according to Item 4.

(Additional Item 6) The endoscope diagnosis support apparatus according to Additional Item 4 or 5, wherein the area calculating means calculates an area based on the number of pixels.

(Additional Item 7) The additional item 2, 3, 4, 5 or 6, wherein the second area setting means and the discriminating and classifying means use different types of characteristic amounts, respectively. Endoscope diagnosis support device.

(Additional Item 8) The additional items 2, 3, wherein the second area setting means sets the second area by area division based on the characteristic amount calculated by the characteristic amount calculating means. 8. The endoscope diagnosis support device according to 4, 5, 6, or 7.

(Additional Item 9) The additional item 2, 3, 4, 5, 6, 7, or wherein the first area set by the first area setting means is the entire endoscope image. 9. The endoscope diagnosis support device according to 8.

(Supplementary item 10) Comprehensive information deriving means for deriving comprehensive diagnosis support information based on the discrimination / classification result of the second area by the discrimination / classification means, and the display means is provided with the comprehensive information derivation. Additional claim 2, characterized in that the diagnostic support information derived by the means is displayed.
10. The endoscope diagnosis support device according to 3, 4, 5, 6, 7, 8, or 9.

(Supplementary Item 11) The comprehensive information deriving means calculates an area ratio obtained by integrating the second regions for each of the determination and classification results, and derives diagnosis support information based on the determination and classification result and the area ratio. 13. The endoscope diagnosis support apparatus according to claim 10, wherein

(Additional Item 12) The endoscope according to Additional Item 11, wherein the comprehensive information deriving means derives diagnosis support information by executing an identification and classification process using the area ratio as a feature amount. Diagnosis support device.

(Additional Item 13) Endoscope image input means for inputting an endoscope image composed of a plurality of color signals, and setting an area for the endoscope image input to the endoscope image input means Area setting means for performing, endoscope image type determination means for determining the type of the endoscope image, feature amount calculation means for calculating a feature amount based on at least one color signal of the endoscope image, A classification unit configured to apply a classification process based on the feature amount calculated by the feature amount calculation unit to the region set by the region setting unit; and a display unit configured to display a result of the classification performed by the classification unit. ,
An endoscope diagnosis support apparatus, wherein the feature amount calculating unit changes a color signal for calculating a feature amount based on a determination result of the endoscope image type determining unit.

(Additional Item 14) An endoscope image type determining means for determining the type of the endoscope image input to the endoscope image input means, and a determination result of the endoscope image type determining means Additional feature 2, wherein the feature value calculating means changes the color signal for calculating the feature value based on
13. The endoscope diagnosis support device according to 3, 4, 5, 6, 7, 9, 10, 11, or 12.

(Additional Item 15) The endoscope diagnosis support according to additional item 13 or 14, wherein the endoscope image type determining means determines the type of the endoscope image based on the presence or absence of the application of the medicine. apparatus.

(Additional Item 16) The endoscope diagnosis support apparatus according to Additional Item 15, wherein the endoscope image type determining means determines the type of image based on the type of the applied medicine.

(Additional Item 17) An additional feature wherein a color signal correcting means for correcting a color signal for calculating the characteristic amount by the characteristic amount calculating means based on the determination result of the endoscope image type determining means is provided. Item 17. The endoscope diagnosis support device according to item 13, 14, 15, or 16.

(Additional Item 18) The additional item 1 characterized in that the color signal correction means performs correction based on inversion of gradation.
An endoscope diagnosis support device according to claim 7.

(Additional Item 19) The additional items 13, 14, 15, and wherein the endoscope image type determining means determines the type of the endoscope image based on the color tone of the endoscopic image.
19. The endoscope diagnosis support device according to 16, 17, or 18.

(Additional Item 20) The additional feature items 2, 3, 4, 5, 6, 7, and wherein the feature value calculating means calculates a feature value based on a spatial frequency component of the endoscope image.
9, 10, 11, 12, 13, 14, 15, 16, 1
20. The endoscope diagnosis support device according to 7, 18, or 19.

(Additional Item 21) The endoscope according to Additional Item 20, wherein the feature amount calculating means calculates a feature amount based on an amplitude and / or a phase in a spatial frequency component of the endoscope image. Diagnosis support device.

(Additional Item 22) The additional item items 2, 3, 4, 5, wherein the discriminating and classifying means applies classification using at least one statistical or non-statistical classifier.
6,7,8,9,10,11,12,13,14,1
The endoscope diagnosis support apparatus according to 5, 16, 17, 18, 19, 20 or 21.

(Additional Item 23) Endoscope image input means for inputting an endoscope image comprising a plurality of color signals, and at least one color of the endoscope image input to the endoscope image input means A spatial frequency component extracting means for extracting a spatial frequency component based on the signal; a phase component detecting means for detecting a phase component of the endoscope image based on the spatial frequency component extracted by the spatial frequency component extracting means; An endoscope image processing apparatus comprising: a feature amount calculating unit that calculates a feature amount based on the phase component detected by the component detecting unit.

(Additional Item 24) The endoscope image processing apparatus according to Additional Item 23, wherein the feature amount calculating method calculates a feature amount based on the direction of the change of the phase component.

(Additional Item 25) The endoscope image processing apparatus according to additional item 23 or 24, wherein the feature value calculating method calculates a feature value based on the magnitude of the change of the phase component.

(Supplementary note 26) An amplitude component detecting means for detecting an amplitude component of the endoscope image based on the spatial frequency component extracted by the spatial frequency component extracting means, and a phase detected by the phase component detecting means. 26. The endoscope image processing apparatus according to claim 23, further comprising: a feature amount calculating unit that calculates a feature amount based on the component and the amplitude component detected by the amplitude component detecting unit. .

(Additional Item 27) The endoscope image processing apparatus according to additional item 26, wherein the characteristic amount calculating means calculates a characteristic amount based on a covariance or a correlation of the phase component and the amplitude component.

(Additional Item 28) Additional items 23 and 2 characterized in that the spatial frequency component extraction means performs Fourier transform or filtering using a digital filter.
28. The endoscope image processing device according to 4, 25, 26 or 27.

(Additional Item 29) A step of inputting an endoscope image, a step of applying filtering for extracting a spatial frequency component to the endoscope image, and a step of applying the endoscope image based on a result of the filtering Detecting the phase information and calculating a feature amount based on the phase information.

(Additional Item 30) An endoscope image input means for inputting an endoscope image composed of a plurality of color signals, and a region is set for the endoscope image input to the endoscope image input means. Region setting means, endoscope image type determining means for determining the type of the endoscope image, and feature amount calculating means for calculating a feature amount based on at least one color signal of the endoscope image, An endoscope image processing apparatus characterized in that the feature amount calculating unit changes a color signal for calculating a feature amount based on a determination result of the endoscope image type determining unit.

(Additional Item 31) A step of inputting an endoscopic image composed of a plurality of color signals, a step of setting an area for the endoscopic image, and a step of determining a type of the endoscopic image Calculating a feature amount based on at least one color signal of the endoscope image,
An endoscope image processing method, comprising: changing a color signal for calculating a feature amount in the feature amount calculation step based on a determination result of the endoscope image type determination unit.

Additional Items 2, 3, 4, 5, 6, 7, 8, 9,
The purpose of 10, 11, 12, and 22 is to show each finding even when the mucosal surface structure of the lesion presents multiple different findings in the endoscopic image, or when other different tumors are mixed in the lesion. In addition to setting a region of interest for each site, it is possible to know what findings or tumors, etc. that are diagnosed in relation to the findings, and what proportion they are mixed It is to provide a diagnosis support device.

Additional Items 1, 3, 1, 4, 1, 5, 1, 6, 1
The purpose of 7, 18, 19, 30, and 31 is to change which of the RGB images is to be processed in the feature value calculation according to the presence or absence of the medicine spraying and the type of the medicine used for the spraying. It is to perform objective and numerical diagnosis with high accuracy.

Additional Items 20, 22, 23, 24, 25, 2
The purpose of 6, 27, 28 and 29 is to use all the information having the spatial frequency component by applying the feature amount calculating method in consideration of the phase information in the feature amount of the spatial frequency component in the endoscope image, It is to perform objective and numerical diagnosis with higher accuracy.

[0255]

As described above, according to the present invention, even when the mucosal surface structure of the lesion shows a plurality of different findings in the endoscopic image, or when the tumor contains other different tumors in the lesion, In addition to setting a region of interest for each site showing each finding, what findings or tumors etc. that are diagnosed in relation to the findings are present, and what proportion they are mixed There is an effect that can be known.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a configuration of an endoscope diagnosis support device according to a first embodiment of the present invention;

FIG. 2 is a functional block diagram for explaining a configuration of a diagnosis support processing execution program in a conventional endoscope diagnosis support apparatus.

FIG. 3 is a flowchart for explaining the operation of a diagnosis support processing execution program in a conventional endoscope diagnosis support apparatus.

FIG. 4 is an explanatory view of an ROI setting window for explaining an ROI setting in a conventional endoscope diagnosis support apparatus.

FIG. 5 is an explanatory view of a report display window for explaining display of diagnosis support information in a conventional endoscope diagnosis support apparatus.

FIG. 6 is a functional block diagram for explaining a configuration of a diagnosis support processing execution program in the endoscope diagnosis support device according to the first embodiment;

FIG. 7 is a flowchart for explaining the operation of a diagnosis support processing execution program in the endoscope diagnosis support apparatus according to the first embodiment;

FIG. 8 is a diagram illustrating a temporary ROI setting window for explaining a temporary ROI setting in the endoscope diagnosis support apparatus according to the first embodiment;

FIG. 9 is a flowchart for explaining the operation of a region dividing process according to the first embodiment;

FIG. 10 is a first explanatory diagram illustrating an area dividing process according to the first embodiment;

FIG. 11 is a second explanatory diagram for explaining the area dividing process according to the first embodiment;

FIG. 12 is an explanatory diagram of a report display window for explaining display of diagnosis support information in the endoscope diagnosis support device according to the first embodiment.

FIG. 13 is a block diagram for explaining a configuration of a diagnosis support processing execution program according to the second embodiment of the present invention;

FIG. 14 is a flowchart for explaining the operation of a diagnosis support execution program according to the second embodiment;

FIG. 15 is a flowchart illustrating derivation of diagnosis support information according to the second embodiment;

FIG. 16 is an explanatory diagram of a report display window for explaining display of diagnosis support information in the endoscope diagnosis support device according to the second embodiment.

FIG. 17 is an explanatory view of a report display window for explaining display of diagnosis support information in the endoscope diagnosis support apparatus according to the third embodiment of the present invention.

FIG. 18 is a block diagram illustrating a configuration of a diagnostic support processing execution program according to a fifth embodiment of the present invention.

FIG. 19 is a flowchart for explaining the operation of a diagnosis support processing execution program according to the fifth embodiment;

FIG. 20 is an explanatory diagram for describing a determination process of a normal image and a stained image according to the fifth embodiment.

FIG. 21 is an explanatory diagram for explaining an operation of a diagnosis support processing execution program 250 according to the sixth embodiment of the present invention.

FIG. 22 is a flowchart for explaining the operation of a diagnosis support processing execution program according to the seventh embodiment of the present invention;

FIG. 23 is an explanatory diagram for describing a stained image.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Diagnosis support apparatus 2 ... Video processor 3, 35 ... Observation monitor 4 ... Input unit 5 ... Server unit 6 ... Conference unit 11 ... A / D converter 12, 32, 50 ... Image processing part 13, 21, 31L ... LAN controller 14, 26, 36 Controller 22 Memory 23, 47 Hard disk 24 Hard disk driver 25 Compressor 33 Expander 34 D / A converter 41 CPU 42 Keyboard 43 Keyboard I / F 44 Search monitor 45 ... Mouse 46 ... Mouse I / F 48 ... Hard disk I / F 49 ... Work memory 50 ... Image processing unit 51 ... Printer 52 ... Printer I / F 61 ... Password storage unit 62 ... Password monitoring unit 63 ... Control restriction unit 80 ... Diagnosis Support processing execution program 81 ... Image input / Management block 82 ... Database management block 83 ... Temporary ROI setting block 84 ... Feature calculation block 85 ... ROI setting block 86 ... Distinction classification block 87 ... Report creation block 88 ... Image processing block

[Procedure amendment]

[Submission date] December 26, 2000 (200.12.
26)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0060

[Correction method] Change

[Correction contents]

The image input / management block 71 is a diagnostic support
The input, endoscopic and retrieval of an endoscope image used in the processing are performed.

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0240

[Correction method] Change

[Correction contents]

(Additional Item 20) The additional feature items 2, 3, 4, 5, 6, 7, and wherein the feature value calculating means calculates a feature value based on a spatial frequency component of the endoscope image.
8, 9, 10, 11, 12, 13, 14, 15, 16,
20. The endoscope diagnosis support device according to claim 17, 18, or 19.

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/18 H04N 7/18 M K F Term (Reference) 2H040 BA00 GA01 GA02 GA10 GA11 4C061 CC06 HH51 LL01 WW08 YY13 5B057 AA07 BA11 CE02 CE09 CH08 DA03 DA12 DA16 DB02 DB06 DB09 DC04 DC25 DC36 5C054 CC07 EA01 EA05 FC12 FC15 FE09 HA12 5L096 AA02 AA06 BA06 CA04 EA05 EA35 FA32 FA33 FA59 GA40 GA55 JA11 JA22

Claims (6)

[Claims] A feature value calculating unit configured to calculate a feature value based on an image signal obtained by capturing an image of an object; and a plurality of feature value data corresponding to each pixel of the image signal output from the feature value calculating unit. Image area extracting means for extracting an area of an image formed by the imaging signal; discriminating and classifying means for performing discrimination and classification of a lesion based on the plurality of feature data corresponding to the image area extracted by the image area extracting means. And a display unit for displaying a result of the determination and classification by the determination and classification unit. 2. The diagnosis support apparatus according to claim 1, wherein said feature amount calculating means includes an initial area setting means for setting initial area data of said imaging signal. 3. The diagnosis support apparatus according to claim 1, wherein said image area extracting means extracts a plurality of image areas. 4. The diagnosis support apparatus according to claim 1, further comprising an area calculation unit that calculates an area of the image region extracted by the image region extraction unit. 5. The diagnosis support apparatus according to claim 3, further comprising an area ratio calculation unit that calculates an area ratio of the plurality of image regions. 6. A feature amount calculating step of calculating a feature amount based on an imaging signal obtained by imaging an object, and based on a plurality of feature amount data corresponding to each pixel of the imaging signal output from the feature amount calculating means. An image area extraction step of extracting an area of an image formed by the imaging signal; a discrimination and classification step of discriminating and classifying a lesion based on the plurality of feature data corresponding to the image area extracted by the image area extraction unit; A display step of displaying a result of the discrimination / classification of the discrimination / classification means.