WO2018159347A1 - Processor device, endoscope system, and method of operating processor device - Google Patents

Abstract

Provided are a processor device, an endoscope system and a method of operating a processor device with which it is possible to assist capture of an image defined by a guideline. An inspection image is acquired by capturing an image of an inspection region in order to obtain a reference corresponding image corresponding to a reference image determined in advance for each inspection region. A reference image selecting unit selects a reference image corresponding to the inspection image from a reference image storage unit which stores the reference images. A similarity calculating unit calculates a degree of similarity between the reference image selected by the reference image selecting unit and the inspection image, from image data of the reference image selected by the reference image selecting unit and image data of the inspection image.

Description

Processor device, endoscope system, and operating method of processor device

The present invention relates to a processor device, an endoscope system, and a method for operating the processor device used in screening performed at a medical examination center or clinic.

In the medical field, diagnosis using an endoscope system including a light source device, an endoscope, and a processor device is widely performed. The endoscope system irradiates illumination light emitted from a light source device to an observation target via the endoscope, and a processor device based on an image signal obtained by imaging the observation target under illumination with the illumination light. An image to be observed is generated. By displaying this image on the monitor, the doctor can make a diagnosis while viewing the image on the monitor.

In recent years, the number of facilities that conduct endoscopy as a screening test is increasing, and even when doctors who perform endoscopy are not necessarily experienced, a high-quality test is required so that a correct diagnosis can be made. Yes. Therefore, guidelines (manuals) have been established for taking images of the inside of the digestive tract at an appropriate level so that objective judgment can be made even if the experience of endoscope is not abundant. (For example, see “Gastroscopic Screening Manual for Countermeasure Screening, 2015 Edition (P56-P63)”). In addition, a double check is performed on whether or not the photographed image is photographed according to the guidelines.

In the guidelines on the examination of the stomach, for example, the following is defined.
・ About 40 frames are required from the pharynx to the duodenum. ・ The order of observation (root) is constant. ・ There is no observation (blind spot) (around the cardia part, most part, after the premise part) Wall / Kominato)
・ How to inflate the stomach is sufficient (constant pressure air supply)
・ Shooting with appropriate brightness ・ Mucous membrane removed ・ Scene including landmarks

However, in order to shoot in accordance with the guidelines, a certain level of skill and experience is required regarding the operation of the endoscope. For example, as shown in FIGS. 25 and 26, in order to take an image of a portion directly under the cardia or in the corner of the stomach as defined in the guideline, an angle operation for greatly rotating the distal end portion of the endoscope is required. The guidelines also require that the lesions hidden between the folds be photographed. In such a case, the pleats are expanded by blowing an air supply gas of appropriate air pressure to check for the presence or absence of the lesions. It is necessary to confirm. As described above, it is not easy for a doctor who is not an expert to perform imaging and operation as defined in the guidelines, and there is a need for assistive technology that can perform imaging according to the guidelines.

On the other hand, as shown in Patent Documents 1 to 3, several techniques are shown. In Patent Document 1, when the number of images defined in the guideline does not reach a specified number, a necessary image is extracted from a moving image. In Patent Document 2, the insertion length of the insertion portion of the endoscope is detected, and a necessary image is automatically taken according to the insertion length. Further, in Patent Document 3, when the number of images in a predetermined examination region does not match the planned number of images to be captured, an alert is generated to notify the user that the image has been missed.

JP 2012-70938 A JP 2012-70937 A JP 2012-70936 A

However, it is difficult to know whether the shooting is performed according to the guidelines only with the assistive technologies of Patent Documents 1 to 3. For example, in Patent Document 1, an image necessary for a guideline is extracted from a moving image. However, if the moving image is not photographed at a position determined by the guideline, the image cannot be extracted. Further, in Patent Document 2, although photographing is automatically performed according to the insertion length, the insertion length can change depending on the physique difference of the patient, so the position where photographing is performed according to the insertion length is determined by the guideline. It may not match the position. Further, in Patent Document 3, even if an attempt is made to retake an image upon receiving an alert, an inexperienced doctor cannot take an image according to the guideline without any support that can be taken according to the guideline. There is a case.

It is an object of the present invention to provide a processor device, an endoscope system, and an operation method of the processor device that can support photographing of an image defined by guidelines.

An endoscope system according to the present invention includes a reference image storage unit that stores a plurality of reference images that are predetermined for each inspection region, and an inspection image obtained by capturing the inspection region in order to obtain a reference equivalent image that corresponds to the reference image. An image acquisition unit that acquires the reference image, a reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image.

It is preferable that the similarity calculation unit calculates the similarity by comparing the noise, structure, or landmark included in the image data of the reference image with the noise, structure, or landmark included in the image data of the inspection image. When the insertion portion of the endoscope is included in the reference image, it is preferable to increase the similarity when the insertion portion is included in the inspection image. The similarity calculation unit calculates a plurality of feature amounts of the reference image and a plurality of feature amounts of the inspection image, calculates a feature amount similarity for each of the calculated reference image feature amount and the inspection image feature amount, and these features The total similarity obtained by combining the quantity similarities is preferably used as the similarity.

It is preferable that the similarity calculation unit calculates the similarity from the scope operation information related to the operation of the endoscope in addition to the image data of the reference image and the image data of the inspection image selected by the reference image selection unit. The scope operation information preferably includes at least one of the angle angle or rotation amount of the endoscope, the distance from the observation target, the insertion length of the insertion portion of the endoscope, and the shape of the insertion portion of the endoscope.

It is preferable to provide a display control unit that displays a warning on the display unit when the similarity does not satisfy a specific condition. When the similarity does not satisfy a specific condition, it is preferable to include a display control unit that displays operation support information related to the operation of the endoscope for the similarity to satisfy the specific condition. As for specific conditions, it is preferable that similarity is a fixed value or more.

The reference image is stored in the reference image storage unit in the RAW data format, and the similarity calculation unit calculates the similarity after applying the specific image processing applied to the inspection image to the reference image in the RAW data format. It is preferable to do. It is preferable that a defect area removing unit that removes the defect area from the inspection image is included, and the similarity calculation unit calculates the similarity between the inspection image from which the defect area has been removed and the reference image.

When an inspection image in a specific inspection area is stored in the storage image storage unit, it is preferable to include a display control unit that displays on the inspection object map that the inspection image in the specific inspection area has been stored. . The inspection image is preferably an image obtained when the still image acquisition instruction unit is operated. An endoscope system comprising the processor device of the present invention described above and a display unit for displaying the degree of similarity.

The operation method of the processor device of the present invention, the image acquisition unit acquires the inspection image by capturing the inspection region in order to obtain the reference equivalent image corresponding to the reference image predetermined for each inspection region, and the reference image The selection unit selects a reference image corresponding to the inspection image from the reference image storage unit that stores the reference image, and the similarity calculation unit selects the image data of the reference image selected by the reference image selection unit and the inspection image. And calculating a similarity between the reference image selected by the reference image selection unit and the inspection image from the image data.

According to the present invention, it is possible to support photographing of an image defined by the guideline.

It is an external view of an endoscope system. It is explanatory drawing which shows the procedure in the case of screening an upper digestive tract. It is explanatory drawing which shows the test | inspection area | region defined in the guideline. It is a block diagram which shows the function of an endoscope system. It is explanatory drawing which shows the scale for a measurement for measuring the insertion length of the insertion part of an endoscope. It is explanatory drawing which shows the relationship between the insertion length of the insertion part of an endoscope, and a test | inspection area | region. It is a table | surface which shows the relationship between the insertion length of the insertion part of an endoscope, and a test | inspection area | region (test | inspection site | part). It is a block diagram which shows the function of the process part for inspection assistance mode. It is a flowchart which shows the method of determining the test | inspection image which permits the comparison with a reference | standard image. It is the table | surface which showed the similarity degree of the reference | standard image and the test | inspection image by the total similarity which combined several feature-value. It is an image figure of the test | inspection assistance image by which the similarity was displayed. It is a flowchart which shows the method of determining whether a test | inspection image is an image along a guideline by similarity. It is an image figure of the test | inspection assistance image in which the warning display is performed. It is an image figure of the inspection assistance image on which operation assistance information was displayed. It is an image figure of the inspection assistance image where the existence of preservation | save of the still image of an inspection image was displayed on the inspection object map. It is an image figure of a test | inspection assistance image when there is a part in a test | inspection area | region. It is an image diagram of an inspection support image in which whether or not a still image of an inspection image is stored is displayed on an inspection object map and scope operation information is displayed. It is an image figure of the test | inspection assistance image in which the reference | standard image was displayed. It is an image figure of a test support image on which a standard image, scope position information, and scope operation information are displayed. It is an image figure of the inspection support image in which the reference image is displayed and whether or not the still image of the inspection image is stored is displayed on the inspection target map. It is an image figure of the test | inspection assistance image on which the similarity with the reference | standard image was displayed. It is an image figure of the test | inspection assistance image in which the warning display is performed with the display of the reference | standard image. It is an image figure of the inspection support image on which operation support information was displayed with the display of a standard image. It is an image figure of the test | inspection assistance image which displays the test image of this time and the last in the same test | inspection area | region. It is a block diagram which shows the function of the process part for automatic imaging | photography modes. It is a table | surface which shows the reference | standard imaging | photography conditions for obtaining 40 reference | standard images defined by the guideline. It is a flowchart which shows the flow of the automatic imaging | photography mode which acquires the still image of a test | inspection image automatically. It is an image figure of a monitor displayed when an inspection image is automatically acquired. It is a flowchart which shows the flow which preserve | saves the still image of the test | inspection image acquired automatically. It is an image figure of the monitor displayed when the still image of a test | inspection image is preserve | saved. It is an image figure of the monitor displayed when the preservation | save of the still image of a test | inspection image was not performed. It is a block diagram which shows the function of the process part for imaging | photography assistance modes. It is a flowchart which shows the flow of the imaging | photography assistance mode which assists the acquisition timing of the still image of a test | inspection image. It is an image figure of the monitor which alert | reports the acquisition timing of a still image. It is a flowchart which shows the flow which preserve | saves a still image when the acquisition timing of a still image is notified. It is explanatory drawing which shows the shape of the endoscope in the case of image | photographing directly under the cardia which is a test | inspection area | region defined by the guideline. It is explanatory drawing which shows the shape of the endoscope in the case of image | photographing the stomach corner | angular part which is a test | inspection area | region defined by the guideline.

[First Embodiment]
As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18 (display unit), and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 has an insertion portion 12a to be inserted into the subject, an operation portion 12b provided at the proximal end portion of the insertion portion 12a, a bending portion 12c and a distal end portion 12d provided at the distal end side of the insertion portion 12a. is doing. By operating the angle knob 13a of the operation portion 12b, the bending portion 12c is bent. By this bending operation, the distal end portion 12d is directed in a desired direction.

In addition to the angle knob 13a, the operation unit 12b includes a still image acquisition instruction unit 13b used for a still image acquisition operation, a mode switching unit 13c used for an observation mode switching operation, and a zoom operation unit used for a zoom magnification change operation. 13d is provided.

The endoscope system 10 has a normal light mode, a special light mode, and an inspection support mode as observation modes. When the observation mode is the normal light mode, normal light such as white light is emitted, and the normal light image is displayed on the monitor 18 based on the image signal obtained by imaging the observation target under illumination with the normal light. indicate. In addition, when the observation mode is the special light mode, special light having a light emission spectrum different from that of normal light is emitted, and a special light image is obtained based on an image signal obtained by imaging the observation target under illumination with the special light. Is displayed on the monitor 18.

Also, when the observation mode is the inspection support mode, the illumination light for inspection is emitted. In order to obtain a reference-equivalent image corresponding to a reference image predetermined for each inspection region in the guideline, an inspection image is acquired by imaging an observation target in the inspection region being illuminated with the inspection illumination light. This inspection image is displayed on the monitor 18. The examination support mode is a mode that assists so that an image defined by the examination guidelines can be surely taken during screening. In this inspection support mode, it is possible to confirm whether or not photographing has been performed according to the guideline by displaying the similarity with the reference image defined in the guideline. The inspection illumination light may be the same as or different from normal light or special light. Further, “corresponding” to the reference image means that the composition of the reference image is similar.

The processor device 16 is electrically connected to the monitor 18 and the user interface 19. The monitor 18 outputs and displays an image to be observed, information attached to the image, and the like. The user interface 19 receives input operations such as designation of a region of interest (ROI: Region Of Interest) and function setting.

When screening the upper digestive tract such as the stomach and esophagus according to the guidelines, as shown in FIG. 2, first, the insertion portion 12a of the endoscope 12 is pushed into the lumen, It is inserted through the esophagus Es and stomach St to the deepest examination position P0 of the duodenum Duo. When the inspection deepest position P0 is reached, acquisition of a still image according to the guideline is started. First, the surgeon operates the still image acquisition instruction unit 13b to capture a still image of the observation image at the examination deepest position P0. Thereafter, a still image is taken in the examination area defined by the guidelines while gradually pulling out the insertion portion 12a of the endoscope 12. When imaging of the duodenum Duo and stomach St in the examination area is completed, the insertion portion 12a is moved to the esophagus Es, and a still image is taken in the examination area of the esophagus Es. When photographing of these still images is completed, the insertion portion 12a is pulled out of the body cavity. In addition, it is preferable to screen in the same procedure also about lower digestive tracts, such as large intestine.

As shown in FIG. 3, there are 13 examination areas A1 to A13 for the duodenum Duo, stomach St, and esophagus Es, for example, as the examination areas defined by the guidelines. A1 and A2 are duodenal duo examination areas, A1 is a portion after the duodenum sucking portion, and A2 is a duodenal bulb portion. A3 to A10 are examination areas of the stomach St, A3 is the anterior part of the pylorus, A4 is the pyloric antrum, A5 is the corner of the stomach, A6 is the lower part of the stomach, A7 is the middle part of the stomach, A8 is the upper part of the stomach, and A9 is the upper part of the stomach The gastric fundus (gastric arch ridge), A10 is the gastric cardia. A11 to A13 are examination areas of the esophagus Es, A11 is the lower part of the esophagus, A12 is the middle part of the esophagus, and A13 is the upper part of the esophagus.

As shown in FIG. 4, the light source device 14 includes a light source unit 20 that emits normal light, special light, and inspection illumination light as illumination light used for illumination of an observation target, and a light source control unit 22 that controls the light source unit 20. It has. The light source unit 20 is a semiconductor light source such as an LED (Light-Emitting Diode), a xenon lamp, or a halogen lamp. In the light source unit 20, when emitting light of a plurality of colors, LEDs are used for a plurality of colors, and a color separation filter for separating broadband light such as a xenon lamp into a plurality of colors is used. The light source control unit 22 controls the light emission amount of the illumination light by turning on / off the LED or the like and adjusting the drive current or drive voltage of the LED or the like. The light source control unit 22 controls the wavelength band of the illumination light by changing the optical filter or the like.

The illumination light emitted from the light source unit 20 is incident on the light guide 24 inserted into the insertion unit 12a through an optical path coupling unit (not shown) formed by a mirror, a lens, or the like. The light guide 24 is incorporated in the endoscope 12 and the universal cord, and propagates illumination light to the distal end portion 12d of the endoscope 12. The universal cord is a cord that connects the endoscope 12 to the light source device 14 and the processor device 16. The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination light propagated through the light guide 24 illuminates the observation target via the illumination optical system 30a.

The imaging optical system 30b has an imaging sensor 32 on which various types of light such as reflected light, scattered light, and fluorescence from an observation target enter. The imaging sensor 32 forms an image to be observed and outputs an image signal. The output image signal is transmitted to the processor device 16. The image sensor 32 includes a monochrome sensor not provided with a color filter in addition to a color image sensor such as an RGB color image sensor or a CMYG color image sensor. As the image sensor 32, a CCD (Charge-Coupled Device) image sensor, a CMOS (Complementary Metal-Oxide Semiconductor) image sensor, or the like can be used.

Further, as shown in FIG. 5, a measurement scale 40 for measuring the insertion length of the insertion portion 12a into the body is provided on the outer peripheral surface of the insertion portion 12a of the endoscope 12. The measurement scale 40 is configured from points provided at a predetermined pitch (for example, in increments of 1 cm) along the longitudinal direction of the insertion portion 12a. The measurement scale 40 is detected by a scale detection sensor 42 provided in a patient's mouth (in the case of an upper endoscope) or anus (in the case of a lower endoscope), and the scale detection sensor 42 is wired to the processor device 16. Alternatively, it is connected wirelessly, and detection information from the scale detection sensor 42 is transmitted to the processor device 16. In FIG. 5, a scale detection sensor 42 is provided on the mouthpiece MP that the patient holds in the mouth.

As shown in FIG. 4, the processor device 16 includes an image signal acquisition unit 50, an image processing unit 52, a display control unit 54, a still image storage control unit 55, a storage image storage unit 56, and scope position information. A calculation unit 57 and a scope operation information calculation unit 58 are provided. An image signal acquisition unit (corresponding to the “image acquisition unit” of the present invention) 50 acquires an image signal corresponding to each observation mode from the endoscope 12. The image processing unit 52 includes a normal light mode processing unit 60, a special light mode processing unit 62, and an inspection support mode processing unit 64. The normal light mode processing unit 60 performs normal light mode image processing such as color conversion processing, color enhancement processing, and structure enhancement processing on the image signal obtained in the normal light mode. As a result, a normal light image having a natural hue can be obtained. The normal light image is input to the display control unit 54. When the still image acquisition instruction unit 13 b is operated, the still image of the normal light image at that time is stored in the storage image storage unit 56 by the still image storage control unit 55.

The special light mode processing unit 62 performs special light mode image processing different from the normal light mode image processing on the image signal obtained in the special light mode to obtain a special light image. The special light mode image processing is the same as the normal light mode processed image and the processing content (color conversion processing, color enhancement processing, structure enhancement processing, etc.), but the processing conditions are different. In the special light image, the visibility of the lesioned part is higher than other parts. The special light image is input to the display control unit 54. When the still image acquisition instruction unit 13 b is operated, the still image of the special light image at that time is stored in the storage image storage unit 56 by the still image storage control unit 55.

The inspection support mode processing unit 64 performs inspection support mode image processing on the image signal obtained in the inspection support mode to obtain an inspection image. The inspection image is input to the display control unit 54. In the image processing for inspection support mode, when the still image acquisition instruction unit 13b is not operated, the inspection image is input to the display control unit 54 as it is. On the other hand, when the still image acquisition instructing unit 13b is operated, an inspection support image for confirming whether or not the photographing is performed according to the guideline is generated. This inspection support image is input to the display control unit 54 together with the inspection image. Then, when the user confirms the inspection support image and the still image acquisition instruction unit 13b is operated, the still image of the inspection image at that time is stored by the still image storage control unit 55 in the storage image storage unit 56. Is remembered. Details of the inspection support mode processing unit 64 will be described later.

The display control unit 54 performs control to display the image input from the image processing unit 52 on the monitor 18. As a result, an image corresponding to the observation mode is displayed on the monitor 18. That is, the normal light image is displayed on the monitor 18 in the normal light mode, and the special light image is displayed on the monitor 18 in the special light mode. In the case of the examination support mode, the examination image is displayed on the monitor 18, and when the still image acquisition instruction unit 13b is operated, the examination assistance image is displayed in addition to the examination image.

The scope position information calculation unit 57 calculates the insertion length of the insertion unit 12a of the endoscope 12 based on the detection information from the scale detection sensor 42. Further, the scope position information calculation unit 57 calculates scope position information indicating in which of the examination regions A1 to A13 the distal end portion 12d of the endoscope 12 is based on the insertion length of the insertion portion 12a. 6 and 7, when the insertion length is between L0 and L1, the scope position information calculation unit 57 is positioned in the inspection area A1 and the insertion length is L1 and L2. If it is in between, the tip 12d is located in the inspection area A2.

When the insertion length is between L2 and L3, the tip 12d is located in the inspection area A3. When the insertion length is between L3 and L4, the distal end portion 12d is located in the inspection area A4. When the insertion length is between L4 and L5, the distal end portion 12d is located in the inspection area A5. When the insertion length is between L5 and L6, the distal end portion 12d is located in the inspection area A6. When the insertion length is between L6 and L7, the distal end portion 12d is located in the inspection area A7. When the insertion length is between L7 and L8, the distal end portion 12d is located in the inspection area A8. When the insertion length is between L8 and L9, the distal end portion 12d is located in the inspection area A9. When the insertion length is between L9 and L10, the distal end portion 12d is located in the inspection area A10.

Further, when the insertion length is between L10 and L11, the distal end portion 12d is located in the inspection region A11. When the insertion length is between L11 and L12, the distal end portion 12d is located in the inspection area A12. When the insertion length is between L12 and L13, the distal end portion 12d is located in the inspection area A13.

Considering that the positional relationship between the insertion length and the examination region differs depending on the physique of the subject, the scope position information calculation unit 57 calibrates the positional relationship between the insertion length and the examination region based on the physique difference. It is preferable. For example, the insertion length at the pyloric region such as the inspection region A3 is compared with the predetermined insertion length of the pyloric part for calibration, and the positional relationship between the insertion length and the inspection region is corrected based on the difference between them. Also good. Further, the distance between the pyloric part such as the examination area A3 and the cardia part of the examination area 10 is compared with the predetermined calibration pylorus and intercardia distance. The positional relationship with the inspection area may be corrected.

The scope operation information calculation unit 58 calculates scope operation information indicating information related to the operation of the endoscope 12. The scope operation information includes the angle angle or rotation amount of the distal end portion 12d of the endoscope 12, the distance from the observation target (observation distance), the insertion length, the shape of the endoscope, and the like. The angle angle or the rotation amount is calculated based on operation information of the operation unit 12b such as an operation amount of the angle knob 13a. The observation distance is calculated based on the average value of the pixel values of the inspection image. The insertion length is calculated by the scope position information calculation unit 57 as described above. The shape of the endoscope is preferably calculated using a magnetic sensor (not shown) provided at the distal end portion 12d of the endoscope. In addition to the distal end portion 12d, a plurality of magnetic sensors may be provided in the insertion portion 12a at equal intervals from the distal end portion 12d.

As shown in FIG. 8, the inspection support mode processing unit 64 operates when the still image acquisition instructing unit 13b is operated in addition to the inspection support mode image processing unit 70 that performs image processing for inspection support mode. As a processing unit, a blurring determination unit 72, a defective area removal unit 73, a reference image selection unit 74, a similarity calculation unit 76, and an inspection support image generation unit 78 are provided. In the inspection support mode, after confirming whether the still image of the inspection image is an image in accordance with the guideline, the still image is stored in the storage image storage unit 56.

Therefore, the still image of the inspection image is shot in two stages: provisional shooting and actual shooting. The default is set to a temporary shooting state, and when the still image acquisition instruction unit 13b is operated, temporary shooting is performed. After the provisional shooting, the still image acquisition instruction unit 13b is further operated to perform the main shooting, and the acquired still image of the inspection image is stored in the storage image storage unit 56. After storing the image in the storage image storage unit 56, the temporary imaging state is restored again.

As shown in FIG. 9, the blur blur determination unit 72 determines whether or not blur or blur has occurred in the still image of the temporarily captured inspection image. The blur blur determination unit 72 calculates a blur blur index value indicating blur or blur of a still image of a temporarily captured inspection image in order to determine blur or blur. As a method of calculating the blur blur index value, for example, there is a method of calculating the blur blur index value based on the spatial frequency of the inspection image because blur and blur and the spatial frequency of the image are related.

The blur blur determination unit 72 determines that blur has occurred in the inspection image when the calculated blur blur index value exceeds the threshold value. Since it is difficult to accurately compare the blurred image with the reference image in this way, guidance for prompting the user to retake the test image is displayed on the monitor 18. On the other hand, when the calculated blur blur index value is equal to or smaller than the threshold value, the blur blur determination unit 72 determines that the blur is not so much as to affect the comparison with the reference image, and the comparison with the reference image is permitted. . In this case, the inspection image is transmitted to the defective area removing unit 73.

The defect area removing unit 73 removes defect areas such as dirt and halation on the observation target from the inspection image. For example, a high pixel area having a pixel value equal to or greater than a certain value in the inspection image is recognized as a halation area, and this area is replaced with the pixel value of the adjacent area. The inspection image after removal of the defective area is transmitted to the reference image selection unit 74.

The reference image selection unit 74 includes a reference image storage unit 74a that stores a plurality of reference images determined in advance for each of the inspection areas A1 to A13. The reference image selection unit 74 performs a comparison process between the reference image stored in the reference image storage unit 74a and the inspection image that has been subjected to the blurring determination or from which the defect area has been removed. The comparison process is performed by pattern matching or the like. Based on this comparison processing, a reference image corresponding to the inspection image is selected from the reference image storage unit 74a. The selected reference image and inspection image are transmitted to the similarity calculation unit 76.

Further, the reference image storage unit 74a stores the reference image in the RAW data format so that the state of the reference image can be changed in accordance with the inspection support mode image processing used by the inspection support mode processing unit 64. Yes. Therefore, the reference image selection unit 74 performs the image processing for inspection support mode (“specific image” of the present invention) on the reference image stored in the reference image storage unit 74a before performing the comparison process with the inspection image. And a comparison process between the reference image subjected to the image processing for the inspection support mode and the inspection image is performed. Note that the reference image selection unit 74 may perform a comparison process between the image before the image processing for the examination support mode and the reference image in the RAW data format. The RAW data is image data that has not been subjected to various types of image processing such as color tone enhancement processing (normal light mode image processing, special light mode image processing, and inspection support mode image processing in this embodiment). Say.

The similarity calculation unit 76 calculates the similarity between the reference image and the inspection image based on the image data of the reference image and the image data of the inspection image. The calculated similarity to the reference image is transmitted to the examination support image generation unit 78. As a method of calculating the similarity, for example, the noise included in the image data of the reference image is compared with the noise included in the image data of the inspection image, and the similarity is higher as the difference in the amount of noise is smaller. . In addition, the structure such as folds included in the image data of the reference image is compared with the structure included in the image data of the inspection image and landmarks such as cardia and pylorus. It is said to be expensive. Moreover, when the insertion part 12a of the endoscope 12 is reflected in the reference image, it is preferable to determine the similarity based on the presence or absence of the insertion part 12a. In this case, when the insertion portion 12a is included in the inspection image, the similarity is increased. Furthermore, the similarity may be calculated not only by comparing the image data of the reference image and the image data of the inspection image but also by adding scope operation information. By adding scope operation information in this way, the similarity can be accurately calculated.

Further, as a method of calculating the similarity, a method using a feature amount such as an image color is conceivable. However, since it is difficult to accurately calculate the similarity with only one feature amount, a plurality of feature amounts are calculated. Using the combined overall similarity, the similarity can be accurately calculated. For example, when using the feature amount A of color, feature amount B of structure, and feature amount C of landmark as the feature amount, the total similarity is calculated by weighting and adding the feature amounts A to C. To do.

Note that the above-described total similarity may be used by the reference image selection unit 74 to select a reference image. For example, as shown in FIG. 10, when there are 1 to 40 reference images, the similarity of the feature amounts A to C with the inspection image for those 40 reference images (in FIG. 10, the similarity is expressed as% (percent)). And the total similarity is calculated based on the similarity of the calculated feature amounts A to C. Then, the reference image with the highest overall similarity is selected. In this case, the reference image having the highest overall similarity is the reference image No. 38, the reference image No. 38 is selected.

The inspection support image generation unit 78 generates an inspection support image based on the similarity between the inspection image and the reference image. The generated examination support image is transmitted to the display control unit 54 and displayed on the monitor 18. As shown in FIG. 11, in the examination support image, the similarity with the reference image (for example, the similarity with the reference image of the pylorus is “80%”) is displayed together with the examination image. As shown in FIG. 12, when the user confirms the similarity between the inspection image and the reference image and confirms that the similarity satisfies a specific condition (for example, a certain value or more), the inspection image is included in the guideline. It is determined that the image is an image along the line (reference equivalent image), and the main shooting is performed again by operating the still image acquisition instruction unit 13b. As a result, the inspection image is stored in the storage image storage unit 56. If the degree of similarity with the reference image does not satisfy a specific condition such as a certain value or more, a warning display 80 may be displayed on the monitor 18 by the display control unit 54 as shown in FIG. Good.

On the other hand, when the user determines that the inspection image is not an image in accordance with the guideline (reference equivalent image), for example, the similarity does not satisfy a specific condition, the still image acquisition instruction unit 13b is not operated for a certain period of time. Alternatively, by operating an inspection image non-adopt button (not shown) provided on the endoscope 12 or the user interface 19, the storage of the inspection image in the storage image storage unit 56 is forgotten. In this case, the display of the inspection support image is stopped and the still image of the inspection image is taken again.

When retaking a still image of an inspection image, the operation of the endoscope 12 is performed so that the user can acquire an inspection image (an inspection image whose similarity satisfies a specific condition) according to the guidelines. It is preferable to display the operation support information for supporting on the monitor 18 by the display control unit 54. The operation support information includes information on how much angle angle and insertion length should be adjusted to obtain an inspection image according to the guideline, and how much air supply gas is blown against the observation target. Information on what to do is included. For example, in the case of FIG. 14, the operation support information 82 regarding the angle angle and the insertion length is displayed. The operation support information may be automatically displayed on the monitor 18 when the similarity with the reference image does not satisfy a specific condition.

In addition, in the inspection support image, in addition to the display of the similarity to the inspection image and the reference image, in each inspection region, display indicating whether the acquisition of the inspection image has succeeded or failed may be performed. As shown in FIG. 15, in the inspection support image, in addition to the similarity between the inspection image and the reference image, the inspection image storage unit 56 stores the inspection image in an inspection object map 90 obtained by mapping the inspection regions A1 to A13. The display control unit 54 displays whether or not the data is stored. In this inspection support image, an indication that the inspection images in the inspection areas A1 and A2 have been saved is displayed (“OK” is displayed in FIG. 15), and still images are acquired for the inspection images in the inspection areas A3 and A4. A display indicating that the image is not stored in the storage image storage unit 56 (“NG” in FIG. 15) is displayed. Here, the OK area and the NG area are preferably displayed in different colors (for example, the OK area is displayed in “blue” and the NG area is displayed in “yellow”). Information about the examination area is acquired from scope position information.

It should be noted that, in the inspection support image, when the presence / absence of storage of the still image of the inspection image is displayed, it is possible to find out whether the inspection image is missed. As shown in FIG. 16, when the inspection images in the inspection areas A1 to A10 and A12 are displayed as saved and the inspection area A11 between A10 and A12 is not displayed, the inspection area A11 is displayed. You can see that he failed to take a picture of (lower esophagus). In such a case, the insertion part 12a of the endoscope 12 is pushed in again, and the examination area A11 (lower esophagus) is imaged.

As shown in FIG. 17, in the examination support image, in addition to the display of the similarity between the examination image and the reference image, information regarding the scope operation information 84 and the scope position information 86 may be displayed. In this inspection support image, the insertion length and the angle angle are displayed as scope operation information, and the reference insertion length and the reference angle angle when the reference image is obtained are also displayed. Further, as the scope position information, an indication as to which inspection region the distal end portion 12d of the endoscope is located is made (for example, inspection region A3). Further, in the examination support image, the shape of the insertion portion 12a of the endoscope 12 in the scope operation information is used to indicate in what shape the insertion portion 12a is inserted in the stomach to be examined. Has been.

[Second Embodiment]
In the first embodiment, the inspection support image displays the similarity to the reference image together with the inspection image. In the second embodiment, as shown in FIG. 18, the similarity to the reference image is displayed. Instead of this, a reference image (for example, a reference image of the pylorus) may be displayed. In this case, the user operates the endoscope 12 so that the inspection image has the same composition as the reference image. When the user has the same structure, the user operates the still image acquisition instruction unit 13b to check the inspection image. The still image is acquired. Further, as shown in FIG. 19, in the examination support image, in addition to the examination image and the reference image, scope operation information 84 and scope position information 86 may be displayed together. In this case, the user operates the endoscope 12 while looking at the scope operation information and matches the structure of the reference image.

Further, as shown in FIG. 20, in the inspection support image, an inspection object map 90 is displayed in addition to the inspection image and the reference image, and the inspection object map 90 stores the inspection image in the storage image storage unit 56. You may make it display about the presence or absence. As with the first embodiment, the display regarding the presence / absence of the storage is performed by displaying OK for the inspection area where the inspection image has been stored in the storage image storage unit 56, and storing the acquired still image of the inspection image. NG is displayed for the inspection area where the storage of the inspection image in the image storage unit 56 is not completed.

Further, as shown in FIG. 21, in the examination support image, in addition to the examination image and the reference image (for example, the reference image of the pylorus), the similarity between the reference image (the similarity with the reference image of the pylorus is “80”). % ") May be calculated and displayed. In this case, by using not only the reference image but also the similarity, an image (reference equivalent image) in accordance with the guideline can be captured more reliably. When the degree of similarity is calculated in this way, in the second embodiment, various types of user support when the degree of similarity does not satisfy a specific condition can be performed.

For example, as shown in FIG. 22, in the inspection support image, in addition to the inspection image, the reference image, and the similarity, a warning display 80 that the similarity does not satisfy a specific condition may be performed. Further, as shown in FIG. 23, operation support information 82 necessary for photographing an inspection image according to the guideline may be displayed in the inspection support image. Thus, by displaying the warning display 80 and the operation support information 82 in addition to the similarity to the reference image, an image (reference equivalent image) in accordance with the guideline can be captured more reliably. Furthermore, an endoscope operation may be supported by adding a function of incorporating a light emitting member such as a light emitting diode (LED) in the angle knob 13a of the operation unit 12b so that the lighting direction can be understood.

In the above embodiment, the degree of similarity with the reference image is displayed or the reference image is displayed so that imaging can be performed according to the guideline in one endoscopic examination. In subsequent follow-up observations, it is preferable to display a plurality of inspection images with different acquisition timings on the monitor 18 so that an effective diagnosis can be made. In this case, in order to allow a lesion to be found in the initial examination and to be followed up after that, when storing the examination image in the storage image storage unit 56 in the initial examination, a scope is used. It is preferable to store the positional information or the scope operation information in association with each other.

As shown in FIG. 24, at the time of follow-up, scope position information or scope operation information at a position where a lesion is found is set in advance, and the tip of the endoscope is located at the position where the lesion is found. When the part 12d reaches, the previous inspection image is displayed together with the inspection image. In addition to the previous examination image, the size of the lesion (polyp) at the time of the current follow-up and the size of the lesion (polyp) at the time of the last follow-up are displayed. Furthermore, a reference image may be displayed.

[Third Embodiment]
In the first and second embodiments, when a still image of an inspection image is acquired by temporary shooting by a user, the similarity with the reference image or the reference image is displayed, and after confirming the similarity and the reference image, Instead of this, the still image of the inspection image is stored in the storage image storage unit, but instead of this, in the third embodiment, the inspection image acquisition timing is displayed during the moving image display of the inspection image. When the obtained imaging conditions at the time of inspection satisfy the reference imaging conditions for obtaining the reference image, provisional imaging is automatically performed to obtain a still image of the inspection image.

In the endoscope system 10 of the third embodiment, instead of the examination support mode, an automatic imaging mode is provided that automatically obtains a still image of an inspection image when the imaging conditions at the time of inspection satisfy the reference imaging conditions. It has been. Therefore, as shown in FIG. 25, in the processor device 16 of the endoscope system 10, an automatic imaging mode processing unit 100 is provided instead of the examination support mode processing unit 64 of the first and second embodiments. ing. Switching to the automatic shooting mode is performed by the mode switching unit 13c.

The automatic shooting mode processing unit 100 includes an automatic shooting mode image processing unit 102 in addition to the reference image selection unit 74, the similarity calculation unit 76, and the examination support image generation unit 78 described in the first and second embodiments. A photographing condition acquisition unit 104 at the time of inspection, a photographing condition determination unit 106, and a still image storage request signal output unit 108. In the third embodiment, since the comparison between the inspection image and the reference image is not performed, the blurring determination unit 72 and the defect area removal unit 73 are not provided, but in addition to the comparison between the imaging condition during inspection and the reference imaging condition. When comparing the inspection image and the reference image, it is preferable to perform the blurring determination unit 72 and the defect area removal unit 73 together.

The automatic photographing mode image processing unit 102 performs automatic photographing mode image processing on the image signal obtained in the automatic photographing mode. The image processing for automatic shooting mode is the same as the image processing for inspection support mode of the first and second embodiments, and an inspection image is obtained by performing this image processing for automatic shooting mode. The inspection image is input to the display control unit 54 and displayed as a moving image on the monitor 18.

The inspection imaging condition acquisition unit 104 acquires the imaging condition when the inspection image is acquired as the inspection imaging condition. The imaging conditions at the time of inspection include at least one of scope position information and scope operation information when an inspection image is acquired. Therefore, every time the inspection imaging condition acquisition unit 104 acquires an inspection image for a predetermined frame, the inspection imaging condition acquisition unit 104 accesses the scope position information calculation unit 57 and the scope operation information calculation unit 58 to acquire the scope position information or the scope operation information. .

The imaging condition determination unit 106 determines whether or not the inspection imaging conditions satisfy the reference imaging conditions for obtaining a reference image. The photographing condition determination unit 106 includes a reference photographing condition storage unit 106a that stores a reference photographing condition for obtaining a reference image defined in the guideline. For example, when there are 1 to 40 reference images, as shown in FIG. As a reference photographing condition for obtaining the reference image No. 1 is stored, and other reference image No. 1 is stored. 2 to reference image No. For reference image No. 40, reference image No. 40 is used as a reference photographing condition for obtaining these reference images. No. 2 reference shooting conditions to reference image No. 40 reference photographing conditions are stored.

Note that the reference photographing conditions include at least one of scope position information and scope operation information for obtaining a reference image. Moreover, the imaging conditions at the time of inspection satisfy the reference imaging conditions include, for example, the following. When the imaging condition is the insertion length, the insertion length Lx when the inspection image is acquired is included in a certain range Ly1 to Ly2 (Ly2> Ly1) including the insertion length Ly for obtaining a predetermined reference image. In addition, the imaging conditions at the time of inspection are said to satisfy the standard imaging conditions.

The still image storage request signal output unit 108, when the imaging condition determination unit 106 determines that the examination imaging condition satisfies any of a plurality of reference imaging conditions stored in the reference imaging condition storage unit 106a. As illustrated in FIG. 27, a still image storage request signal for requesting storage of the inspection image in the storage image storage unit 56 is output to the display control unit 54 or the still image storage control unit 55. Note that when the imaging condition determination unit 106 determines that the imaging conditions at the time of the inspection do not satisfy any of the reference imaging conditions, the user holds the endoscope 12 so as to be positioned at the inspection site defined in the guideline. Operate to reacquire the imaging conditions at the time of inspection.

Upon receiving the still image storage request signal, the display control unit 54 displays on the monitor 18 a confirmation message 120 for confirming whether or not to save the inspection image as a still image, as shown in FIG. Let When the still image storage request signal is output, the reference image selection unit 74 selects a reference image corresponding to the reference imaging condition that satisfies the imaging conditions at the time of inspection from the reference image storage unit 74a. The selected reference image is displayed on the monitor 18 by the display control unit 54 (“reference image of the pylorus” is displayed in FIG. 28). In the reference image storage unit 74a of the third embodiment, the reference image is stored in association with the reference shooting condition so that the reference image can be selected from the reference shooting condition.

Also, the similarity calculation unit 76 calculates the similarity between the reference image selected by the reference image selection unit 74 and the inspection image when the still image storage request signal is output. The calculated similarity is displayed on the monitor 18 by the display control unit 54 (in FIG. 28, “similarity with the reference image of the pylorus (80%)” is displayed).

As shown in FIG. 29, the still image storage control unit 55 receives a still image storage request signal, and the user selects an image (reference equivalent image) of the inspection image displayed on the monitor 18 in accordance with the guideline. When the operation of the still image acquisition instruction unit 13b is performed, control for storing the still image of the inspection image in the storage image storage unit 56 is performed. When the still image of the inspection image is stored, as shown in FIG. 30, the display control unit 54 displays on the monitor 18 with a storage completion message 122 indicating that the still image has been stored. Further, the display control unit 54 displays the inspection area where the still image is stored on the inspection object map 90. In this inspection object map 90, “OK” is displayed in the inspection area where the still image is stored among the inspection areas A1 to A13.

On the other hand, although the still image storage control unit 55 has received the still image storage request signal, the user selects an image in which the inspection image displayed on the monitor 18 is an inspection image still image in accordance with the guideline (reference equivalent image). ), And if the operation of the still image acquisition instruction unit 13b is not performed within a predetermined time from the reception of the still image storage request signal, the inspection image is stored in the storage image storage unit 56. I will not. Even when a still image storage request signal is received but an inspection image non-adopting button (not shown) provided on the endoscope 12 or the user interface 19 is operated, the inspection image storage image storage unit 56 is stored. Saving to is not done. If the still image of the inspection image acquired automatically is not stored, it may be automatically switched to the manual photographing mode in which the acquisition of the still image of the inspection image is performed manually. In this case, it is preferable to display on the monitor 18 that the manual photographing mode has been switched.

As described above, when the inspection image is not stored in the storage image storage unit 56, the display of the confirmation message 120 for confirming whether or not the inspection image is to be stored is also stopped. Instead of the confirmation message 120, as shown in FIG. 31, operation support information 82 necessary for photographing an image (reference equivalent image) in accordance with the guideline is displayed on the monitor 18. Then, the user operates the endoscope 12 while confirming the operation support information 82 on the monitor 18 and reacquires the imaging conditions at the time of examination.

[Fourth Embodiment]
In the third embodiment, the still image of the inspection image is automatically acquired when the inspection imaging condition satisfies the reference imaging condition. However, in the fourth embodiment, the inspection imaging condition is the reference imaging. When the condition is satisfied, it is notified that it is the acquisition timing of the still image of the inspection image, that is, the shutter timing is notified, thereby assisting the user to acquire (capture) the still image of the inspection image.

In the endoscope system 10 according to the fourth embodiment, instead of the inspection support mode according to the first and second embodiments, the acquisition timing of the still image of the inspection image is notified when the inspection imaging condition satisfies the reference imaging condition. A shooting assist mode is provided. Therefore, as shown in FIG. 32, in the processor device 16 of the endoscope system 10, a photographing assist mode processing unit 200 is provided instead of the examination support mode processing unit 64 of the first and second embodiments. ing. Note that switching to the shooting assist mode is performed by the mode switching unit 13c. In the fourth embodiment, in addition to the shooting assist mode, the automatic shooting mode of the third embodiment may be provided, and the mode may be switched between the shooting assist mode and the automatic shooting mode.

The imaging assist mode processing unit 200 includes the reference image selection unit 74, the similarity calculation unit 76, and the inspection support image generation unit 78 shown in the first and second embodiments, and the imaging at the time of inspection shown in the third embodiment. In addition to the condition acquisition unit 104 and the shooting condition determination unit 106, a shooting assist mode image processing unit 202 and a still image acquisition timing notification unit 204 are provided. In the fourth embodiment, since the comparison between the inspection image and the reference image is not performed, the blurring determination unit 72 and the defect area removal unit 73 are not provided, but in addition to the comparison between the inspection imaging condition and the reference imaging condition. When comparing the inspection image and the reference image, it is preferable to perform the processing by the blurring determination unit 72 or the defect region removal unit 73 together.

The imaging assist mode image processing unit 202 performs imaging assist mode image processing on the image signal obtained in the imaging assist mode. The imaging assist mode image processing is the same as the inspection assistance mode image processing of the first and second embodiments, and an inspection image is obtained by performing the imaging assist mode image processing. The inspection image is input to the display control unit 54 and displayed as a moving image on the monitor 18.

Also in the fourth embodiment, as in the third embodiment, the imaging condition when acquiring the inspection image is acquired by the in-inspection imaging condition acquisition unit 104 as the in-inspection imaging condition. Then, the imaging condition determination unit 106 determines whether or not the acquired imaging condition at the time of inspection satisfies the reference imaging condition. The imaging conditions at the time of inspection include at least one of the scope position information and the scope operation information when the inspection image is acquired. Therefore, every time an inspection image for a predetermined frame is obtained, the scope position information calculation unit 57 and The scope operation information calculation unit 58 is accessed to acquire scope position information or scope operation information.

As shown in FIG. 33, the still image acquisition timing notification unit 204 is the acquisition timing of the still image of the inspection image when the imaging condition determination unit 106 determines that the inspection imaging condition satisfies the reference imaging condition. Is notified. Note that when the imaging condition determination unit 106 determines that the imaging conditions at the time of the inspection do not satisfy any of the reference imaging conditions, the user holds the endoscope 12 so as to be positioned at the inspection site defined in the guideline. Operate to reacquire the imaging conditions at the time of inspection.

As shown in FIG. 34, a still image acquisition timing, that is, a shutter timing notification 210 is displayed on the monitor 18 via the display control unit 54. Further, when the acquisition timing of the still image is notified, the reference image corresponding to the reference imaging condition that satisfies the imaging condition at the time of inspection is selected by the reference image selection unit 74 as in the third embodiment. The selected reference image (“reference image of the pylorus” in FIG. 34) is displayed on the monitor 18. In addition, the similarity calculation unit 76 calculates the similarity between the reference image selected by the reference image selection unit 74 and the inspection image. The calculated similarity (“Similarity with the reference image of the pylorus (80%)” in FIG. 34) is also displayed on the monitor 18. The still image acquisition instructing unit 13b in the operation unit 12b is configured with an LED (Light Emitting （Diode) built-in button or the like, and the LED built in the still image acquisition instructing unit 13b is turned on at the time of acquiring a still image The acquisition timing of the still image may be notified.

As shown in FIG. 35, the still image storage control unit 55 is notified of the acquisition timing of the still image, and the user follows the guideline that the still image of the inspection image displayed on the monitor 18 is the still image of the inspection image. When it is determined that the image is an image (reference equivalent image) and the still image acquisition instruction unit 13b is operated, the still image of the inspection image is stored in the storage image storage unit 56 as in the third embodiment. Control to memorize. When the still image of the inspection image is stored, it is preferable to display the inspection region where the still image is stored on the inspection object map 90 as in the third embodiment (see FIG. 30). ).

On the other hand, although the still image storage control unit 55 has been notified of the acquisition timing of the still image, the user can obtain a still image of the inspection image displayed on the monitor 18 and a still image of the inspection image in accordance with the guideline (corresponding to the reference). When the operation of the still image acquisition instructing unit 13b is not performed within a certain time from the notification (in addition, when the examination image non-adopting button shown in the third embodiment is operated) In this case, the still image of the inspection image is not saved in the saving image storage unit 56. In this case, the user operates the endoscope 12 so as to be positioned at the examination site defined in the guideline, and reacquires the examination imaging conditions. When the still image is not stored, it is preferable to display the operation support information 82 necessary for photographing an image (reference equivalent image) in accordance with the guideline on the monitor 18 as in the third embodiment ( (See FIG. 31).

In the above embodiment, the hardware structure of a processing unit (processing unit) that executes various types of processing such as the image processing unit 52 is the following various types of processors. For various processors, the circuit configuration is changed after the manufacture of CPU (Central Processing Unit) and FPGA (Field Programmable Gate Array), which are general-purpose processors that function as various processing units by executing software (programs). A programmable logic device (Programmable Logic Device: PLD) that is a possible processor, a dedicated electric circuit that is a processor having a circuit configuration specifically designed to execute various processes, and the like are included.

One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or CPUs and FPGAs). May be. Further, the plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or server, one processor is configured with a combination of one or more CPUs and software, There is a form in which this processor functions as a plurality of processing units. Second, as represented by a system-on-chip (SoC), a form of using a processor that realizes the functions of the entire system including a plurality of processing units with a single IC (integrated circuit) chip. is there. As described above, various processing units are configured using one or more of the various processors as a hardware structure.

Furthermore, the hardware structure of these various processors is more specifically an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

DESCRIPTION OF SYMBOLS 10 Endoscope system 12 Endoscope 12a Insertion part 12b Operation part 12c Bending part 12d Tip part 13a Angle knob 13b Still image acquisition instruction part 13c Mode switching part 13d Zoom operation part 14 Light source apparatus 16 Processor apparatus 18 Monitor 19 User interface 20 Light source unit 22 Light source control unit 24 Light guide 30a Illumination optical system 30b Imaging optical system 32 Imaging sensor 42 Detection sensor 50 Image signal acquisition unit 52 Image processing unit 54 Display control unit 55 Still image storage control unit 56 Storage image storage unit 57 Scope Position information calculation unit 58 Scope operation information calculation unit 60 Normal light mode processing unit 62 Special light mode processing unit 64 Inspection support mode processing unit 70 Inspection support mode image processing unit 72 Blur blur determination unit 73 Defective area removal unit 74 Standard Image selection unit 74a Reference image storage unit 76 Similarity calculation unit 78 Inspection support image generation unit 80 Warning display 82 Operation support information 84 Scope operation information 86 Scope position information 90 Inspection object map 100 Automatic imaging mode processing unit 102 Automatic imaging mode image processing unit 104 Acquisition of imaging conditions during inspection Unit 106 shooting condition acquisition unit 108 reference shooting condition storage unit 108 still image storage request signal output unit 120 confirmation message 122 storage completion message 200 shooting assist mode processing unit 202 shooting assist mode image processing unit 204 still image acquisition timing notification unit 210 Notification of shutter timing

Claims (15)

1. A reference image storage unit that stores a plurality of reference images predetermined for each inspection region;
   An image acquisition unit that captures the inspection region and obtains an inspection image in order to obtain a reference-equivalent image corresponding to the reference image;
   A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit;
   A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image; A processor device comprising:
2. The similarity calculation unit calculates the similarity by comparing noise, structure, or landmark included in image data of the reference image with noise, structure, or landmark included in image data of the inspection image. The processor device according to 1.
3. 3. The processor device according to claim 1, wherein when the reference image includes an insertion portion of an endoscope, the degree of similarity is increased when the inspection image includes the insertion portion.
4. The similarity calculating unit calculates a plurality of feature amounts of the reference image and a plurality of feature amounts of the inspection image, and calculates a feature amount similarity for each of the calculated reference image feature amount and the inspection image feature amount. 4. The processor device according to claim 1, wherein an overall similarity obtained by combining these feature quantity similarities is used as the similarity.
5. The similarity calculation unit calculates the similarity from scope operation information related to an operation of an endoscope in addition to image data of the reference image selected by the reference image selection unit and image data of the inspection image. Item 5. The processor device according to any one of Items 1 to 4.
6. The scope operation information includes at least one of an angle angle or rotation amount of an endoscope, a distance from an observation target, an insertion length of the insertion portion of the endoscope, and a shape of the insertion portion of the endoscope. Item 6. The processor device according to Item 5.
7. 7. The processor device according to claim 1, further comprising a display control unit that displays a warning on the display unit when the similarity does not satisfy a specific condition.
8. 8. The display control unit according to claim 1, further comprising: a display control unit configured to display operation support information related to an operation of an endoscope for satisfying the specific condition when the similarity does not satisfy a specific condition. The processor device according to claim 1.
9. The processor device according to claim 7 or 8, wherein the specific condition is that the similarity is a certain value or more.
10. The reference image is stored in the reference image storage unit in a RAW data format,
    The processor according to any one of claims 1 to 9, wherein the similarity calculation unit calculates the similarity after performing specific image processing performed on the inspection image on a reference image in the RAW data format. apparatus.
11. A defect area removing unit for removing the defect area from the inspection image;
    11. The processor device according to claim 1, wherein the similarity calculation unit calculates a similarity between the inspection image from which the defect area is removed and the reference image.
12. A display control unit that displays on the inspection object map that the inspection image in the specific inspection area is already stored when the inspection image in the specific inspection area is stored in the storage image storage unit. The processor device according to any one of 1 to 11.
13. The processor device according to any one of claims 1 to 12, wherein the inspection image is an image obtained when a still image acquisition instruction unit is operated.
14. A processor device according to any one of claims 1 to 13,
    An endoscope system comprising a display unit for displaying the similarity.
15. The image acquisition unit captures the inspection area and acquires an inspection image to obtain a reference-equivalent image corresponding to a reference image predetermined for each inspection area; and
    A reference image selection unit selecting a reference image corresponding to the inspection image from a reference image storage unit storing the reference image;
    The similarity calculation unit calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image Operating the processor device.

Images