JP6774552B2 - Processor device, endoscopic system and how to operate the processor device - Google Patents

Description

The present invention relates to a processor device, an endoscopic system, and a method of operating the processor device used in screening performed in a screening center or clinic.

In the medical field, diagnosis using an endoscope system including a light source device, an endoscope, and a processor device is widely performed. In the endoscope system, the processor device irradiates the observation target with the illumination light emitted by the light source device through the endoscope, and the processor device is based on the image signal obtained by imaging the observation target under illumination with the illumination light. Generate an image to be observed. By displaying this image on the monitor, the doctor can make a diagnosis while looking at the image on the monitor.

In recent years, the number of facilities that perform endoscopy as a screening test is increasing, and even if the doctor who performs endoscopy is not necessarily experienced, high-quality tests are required so that a correct diagnosis can be made. There is. Therefore, guidelines (manuals) have been established for taking images of the entire inside of the digestive tract at an appropriate level so that even if you do not have a lot of experience with endoscopes, you can make objective judgments. (For example, refer to "Gastroscopy Examination Manual for Countermeasure Examination 2015 Edition (P56-P63)"). In addition, the captured image is then double-checked to see if it was captured according to the guidelines.

The guidelines for gastric examination, for example, stipulate the following.
・ It is necessary to take about 40 frames from the pharynx to the duodenum. ・ Keep the order of observation (route) constant. ・ There are no unobserved areas (blind spots) (all around the cardia, most of the area behind the premise). Wall / Lesser curvature)
・ Stomach swelling is sufficient (constant pressure insufflation)
・ The image was taken with appropriate brightness ・ The mucous membrane was removed ・ The scene including landmarks was done

However, in order to take pictures according to the guidelines, a certain level of skill and experience is required for the operation of the endoscope. For example, as shown in FIGS. 25 and 26, when an attempt is made to photograph a portion directly under the cardia or a portion of the gastric incisure defined in the guideline, an angle operation for greatly rotating the tip portion of the endoscope is required. In addition, the guidelines also require that the lesions hidden between the folds be photographed. In such cases, the folds are widened by blowing air pressure of appropriate air pressure to show the presence of lesions. It is necessary to confirm. In this way, it is not easy for an inexperienced doctor, even if he is not a specialist, to perform the imaging and operation specified in the guideline, and assistive technology that enables imaging according to the guideline has been required.

On the other hand, as shown in Patent Documents 1 to 3, some techniques are shown. In Patent Document 1, when the number of images defined by the guideline does not reach the specified number, a necessary image is extracted from the moving image. Further, in Patent Document 2, the insertion length of the insertion portion of the endoscope is detected, and a necessary image is automatically taken according to the insertion length. Further, in Patent Document 3, when the number of images at a predetermined inspection site does not match the planned number of images to be imaged, an alert is generated to notify the user of the failure to take the image.

Japanese Unexamined Patent Publication No. 2012-702938 Japanese Unexamined Patent Publication No. 2012-70937 Japanese Unexamined Patent Publication No. 2012-70936

However, it is difficult to know whether or not the photographing is performed according to the guideline only by the assistive technologies of Patent Documents 1 to 3. For example, in Patent Document 1, an image necessary for a guideline is extracted from a moving image, but if the moving image is not taken at a position determined by the guideline, the image cannot be extracted. Further, in Patent Document 2, imaging is automatically performed according to the insertion length, but since the insertion length may change depending on the physical disparity of the patient, the position of imaging according to the insertion length is defined by the guideline. It may not match the position. Further, in Patent Document 3, even if an alert is received and an attempt is made to take a picture again, an inexperienced doctor cannot take a picture according to the guideline without some support. In some cases.

An object of the present invention is to provide a processor device, an endoscope system, and a method of operating the processor device that can support the acquisition of an image defined by a guideline.

The processor device of the present invention has a reference image storage unit that stores a plurality of reference images predetermined for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference The color feature amount, the structural feature amount and / or the landmark feature amount related to the reference image and the inspection image selected by the image selection unit are calculated, and the color feature amount and the structure related to the reference image and the inspection image are calculated. The feature amount similarity is calculated for each feature amount and / or the feature amount of the landmark, weighted addition is performed for each feature amount similarity degree, and the total similarity degree obtained by combining multiple feature amount similarity is calculated as the similarity degree . comprises a similarity calculating unit, when there is no is was satisfy the specific condition similarity, and a display control unit for warning display to the display unit.

The processor device of the present invention has a reference image storage unit that stores a plurality of predetermined reference images for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference A similarity calculation unit that calculates the similarity between the reference image and the inspection image selected by the image selection unit, and a display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition. When the reference image includes the insertion part of the endoscope, the similarity is increased when the inspection image includes the insertion part.

The processor device of the present invention has a reference image storage unit that stores a plurality of predetermined reference images for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference A similarity calculation unit that calculates the similarity between the reference image and the inspection image selected by the image selection unit, and a display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition. The similarity calculation unit is a reference image selected by the reference image selection unit from the image data of the reference image selected by the reference image selection unit, the image data of the inspection image, and the scope operation information related to the operation of the endoscope. And the inspection image are calculated.
The processor device of the present invention has a reference image storage unit that stores a plurality of predetermined reference images for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference A similarity calculation unit that calculates the similarity between the reference image and the inspection image selected by the image selection unit, and a display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition. The similarity calculation unit is a reference image selected by the reference image selection unit from the image data of the reference image selected by the reference image selection unit, the image data of the inspection image, and the scope operation information related to the operation of the endoscope. The similarity between the image and the inspection image is calculated, and the scope operation information includes the angle or rotation amount of the endoscope, the distance to the observation target, the insertion length of the insertion part of the endoscope, and the insertion part of the endoscope. At least one of the shapes is included.

The processor device of the present invention has a reference image storage unit that stores a plurality of predetermined reference images for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference A similarity calculation unit that calculates the similarity between the reference image and the inspection image selected by the image selection unit, and a display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition. The display control unit displays operation support information regarding the operation of the endoscope so that the similarity satisfies a specific condition.
The processor device of the present invention has a reference image storage unit that stores a plurality of predetermined reference images for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference A similarity calculation unit that calculates the similarity between the reference image and the inspection image selected by the image selection unit, and a display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition. The reference image storage unit stores the reference image in the RAW data format, and the similarity calculation unit applies the image data of the reference image and the image data of the inspection image selected by the reference image selection unit to the inspection image. After applying the specific image processing performed to the reference image in the RAW data format, the similarity between the reference image selected by the reference image selection unit and the inspection image is calculated.

The processor device of the present invention has a reference image storage unit that stores a plurality of predetermined reference images for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference A similarity calculation unit that calculates the similarity between the reference image and the inspection image selected by the image selection unit, and a display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition. A defect area removing unit for removing a defect area from the inspection image is provided, and the similarity calculation unit is selected by the reference image selection unit from the image data of the reference image and the image data of the inspection image selected by the reference image selection unit. The degree of similarity between the reference image obtained and the inspection image from which the defective region has been removed is calculated.
The processor device of the present invention has a reference image storage unit that stores a plurality of reference images predetermined for each inspection area, and an inspection area is imaged to acquire an inspection image in order to obtain a reference equivalent image corresponding to the reference image. From the image acquisition unit, the reference image selection unit that selects the reference image corresponding to the inspection image from the reference image storage unit, the image data of the reference image selected by the reference image selection unit, and the image data of the inspection image, the reference A similarity calculation unit that calculates the similarity between the reference image and the inspection image selected by the image selection unit, and a display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition. When the inspection image in the specific inspection area is stored in the storage image storage unit, the display control unit displays on the inspection target map that the inspection image in the specific inspection area has been saved. ..

The similarity calculation unit preferably calculates the similarity by comparing the noise, structure or landmark contained in the image data of the reference image with the noise, structure or landmark contained in the image data of the inspection image. Under specific conditions, the similarity is preferably a certain value or more. The inspection image is preferably an image obtained when the still image acquisition instruction unit is operated. An endoscope system including the processor device of the present invention described above and a display unit for displaying the degree of similarity. It is preferable that the endoscope system includes a processor device and a display unit for displaying the similarity.

Method of operating a processor system of the present invention includes the steps of image acquisition unit acquires an inspection image by imaging the inspection area in order to obtain a reference corresponding image corresponding to the predetermined reference image for each inspection area, the reference The image selection unit selects a reference image corresponding to the inspection image from the reference image storage unit that stores the reference image, and the similarity calculation unit inspects the image data of the reference image selected by the reference image selection unit. From the image data of the image, the color feature amount, the structural feature amount and / or the landmark feature amount related to the reference image and the inspection image selected by the reference image selection unit are calculated, and the reference image and the inspection image are related. The feature amount similarity is calculated for each of the color feature amount, the structure feature amount, and / or the landmark feature amount, and the weighting addition is performed for each feature amount similarity, and the overall similarity is a combination of a plurality of feature amount similarity. It has a step of calculating the degree as the degree of similarity and a step of causing the display control unit to display a warning to the display unit when the degree of similarity does not satisfy a specific condition.

According to the present invention, it is possible to support the taking of an image defined by the guideline.

It is an external view of an endoscope system. It is explanatory drawing which shows the procedure when screening the upper gastrointestinal tract. It is explanatory drawing which shows the inspection area defined in the guideline. It is a block diagram which shows the function of an endoscope system. It is explanatory drawing which shows the scale for measurement for measuring the insertion length of the insertion part of an endoscope. It is explanatory drawing which shows the relationship between the insertion length of the insertion part of an endoscope and an examination area. It is a table which shows the relationship between the insertion length of the insertion part of an endoscope and the inspection area (inspection site). It is a block diagram which shows the function of the processing part for inspection support mode. It is a flowchart which shows the method of determining the inspection image which allows comparison with a reference image. It is a table which showed the similarity of a reference image and an inspection image by the total similarity which combined a plurality of feature quantities. It is an image diagram of the inspection support image which displayed the degree of similarity. It is a flowchart which shows the method of determining whether or not the inspection image is an image according to a guideline by the degree of similarity. It is an image diagram of an inspection support image in which a warning is displayed. It is an image diagram of the inspection support image which displayed the operation support information. It is an image diagram of the inspection support image which shows whether or not the still image of the inspection image is saved on the inspection target map. It is an image diagram of the inspection support image when there is a leftover in a part of inspection areas. It is an image diagram of the inspection support image which displays whether or not the still image of the inspection image is saved on the inspection target map, and also displays the scope operation information. It is an image diagram of the inspection support image which displayed the reference image. It is an image diagram of the inspection support image which displayed the reference image, the scope position information, and the scope operation information. It is an image diagram of an inspection support image in which a reference image is displayed and whether or not a still image of the inspection image is saved is displayed on an inspection target map. It is an image diagram of the inspection support image which displayed the similarity with the reference image. It is an image diagram of an inspection support image in which a warning is displayed together with a reference image. It is an image diagram of the inspection support image in which the operation support information is displayed together with the display of the reference image. It is an image diagram of the inspection support image which displays the inspection image of this time and the last inspection in the same inspection area. It is a block diagram which shows the function of the processing part for an automatic shooting mode. It is a table which shows the standard photographing condition for obtaining 40 standard images defined in the guideline. It is a flowchart which shows the flow of the automatic shooting mode which automatically acquires a still image of an inspection image. It is an image diagram of the monitor displayed when the inspection image is automatically acquired. It is a flowchart which shows the flow which saves the still image of the inspection image acquired automatically. It is an image diagram of the monitor displayed when the still image of the inspection image is saved. It is an image diagram of the monitor which is displayed when the still image of the inspection image is not saved. It is a block diagram which shows the function of the processing part for a shooting assist mode. It is a flowchart which shows the flow of the shooting assist mode which assists the acquisition timing of the still image of an inspection image. It is an image diagram of a monitor which notifies the acquisition timing of a still image. It is a flowchart which shows the flow which saves a still image when the acquisition timing of a still image is notified. It is explanatory drawing which shows the shape of an endoscope at the time of photographing just under the cardia which is an inspection area defined in the guideline. It is explanatory drawing which shows the shape of the endoscope at the time of photographing the gastric angle part which is the examination area defined in the guideline.

[First Embodiment]
As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18 (display unit), and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 has an insertion portion 12a to be inserted into the subject, an operation portion 12b provided at the base end portion of the insertion portion 12a, and a curved portion 12c and a tip portion 12d provided on the tip end side of the insertion portion 12a. doing. By operating the angle knob 13a of the operation unit 12b, the bending unit 12c bends. By this bending operation, the tip portion 12d is directed in a desired direction.

In addition to the angle knob 13a, the operation unit 12b includes a still image acquisition instruction unit 13b used for still image acquisition operation, a mode switching unit 13c used for observation mode switching operation, and a zoom operation unit used for zoom magnification changing operation. 13d is provided.

The endoscope system 10 has a normal light mode, a special light mode, and an inspection support mode as observation modes. When the observation mode is the normal light mode, the normal light image such as white light is emitted, and the normal light image is transmitted to the monitor 18 based on the image signal obtained by imaging the observation target under illumination with the normal light. indicate. Further, when the observation mode is the special light mode, a special light having a different emission spectrum from the normal light is emitted, and a special light image is obtained based on an image signal obtained by imaging an observation target under illumination with this special light. Is displayed on the monitor 18.

Further, when the observation mode is the inspection support mode, the inspection illumination light is emitted. Then, in order to obtain a reference equivalent image corresponding to the reference image predetermined for each inspection area in the guideline, the observation target in the inspection area under illumination is imaged with the inspection illumination light to acquire the inspection image. This inspection image is displayed on the monitor 18. The examination support mode is a mode that assists in surely taking an image defined by a guideline for examination at the time of screening. In this inspection support mode, it is possible to confirm whether or not the image is taken according to the guideline by displaying the degree of similarity with the reference image defined in the guideline. The inspection illumination light may be the same as the normal light or the special light, or may be different. Further, "corresponding" to the reference image means that the composition of the reference image is similar.

The processor device 16 is electrically connected to the monitor 18 and the user interface 19. The monitor 18 outputs and displays an image to be observed, information incidental to the image, and the like. The user interface 19 accepts input operations such as designation of a region of interest (ROI) and function settings.

When screening the upper gastrointestinal tract such as the stomach and esophagus according to the guideline, first, as shown in FIG. 2, the insertion portion 12a of the endoscope 12 is pushed into the lumen. It is inserted through the esophagus Es and stomach St to the deepest position P0 of the duodenal Duo examination. When the inspection deepest position P0 is reached, acquisition of a still image according to the guideline is started. First, the operator operates the still image acquisition instruction unit 13b to take a still image of the observation image at the deepest inspection position P0. After that, while gradually pulling out the insertion portion 12a of the endoscope 12, a still image is taken in the inspection area defined by the guideline. After the imaging of the duodenal Duo and the stomach St in the examination area is completed, the insertion portion 12a is then moved to the esophagus Es, and a still image is taken in the examination area of the esophagus Es. When the shooting of these still images is completed, the insertion portion 12a is pulled out of the body cavity. It is preferable to perform screening for the lower gastrointestinal tract such as the large intestine by the same procedure.

As the inspection area defined by the guideline, as shown in FIG. 3, for example, there are 13 inspection areas A1 to A13 for duodenum Duo, stomach St, and esophagus Es. A1 and A2 are examination areas of duodenal Duo, A1 is below the posterior duodenal suction, and A2 is the duodenal bulb. A3 to A10 are the examination areas of the stomach St, A3 is the anterior pylorus, A4 is the antrum of the pylorus, A5 is the corpus, A6 is the lower body of the stomach, A7 is the middle body of the stomach, A8 is the upper body of the stomach, and A9 is the upper body of the stomach. The bottom of the stomach (the ridge of the gastric arch), A10 is the gastric pylorus. A11 to A13 are examination areas of the esophagus Es, A11 is the lower part of the esophagus, A12 is the middle part of the esophagus, and A13 is the upper part of the esophagus.

As shown in FIG. 4, the light source device 14 includes a light source unit 20 that emits normal light, special light, and inspection illumination light as illumination light used for illuminating an observation target, and a light source control unit 22 that controls the light source unit 20. It has. The light source unit 20 is a semiconductor light source such as an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp. In the light source unit 20, when emitting light of a plurality of colors, LEDs are used for a plurality of colors, and a color separation filter for separating wideband light such as a xenon lamp into a plurality of colors is used. The light source control unit 22 controls the amount of light emitted from the illumination light by turning on / off the LED and the like, and adjusting the drive current and the drive voltage of the LED and the like. Further, the light source control unit 22 controls the wavelength band of the illumination light by changing the optical filter or the like.

The illumination light emitted by the light source unit 20 enters the light guide 24 inserted into the insertion unit 12a via an optical path coupling portion (not shown) formed by a mirror, a lens, or the like. The light guide 24 is built in the endoscope 12 and the universal cord, and propagates the illumination light to the tip portion 12d of the endoscope 12. The universal cord is a cord that connects the endoscope 12, the light source device 14, and the processor device 16. The tip portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination light propagating through the light guide 24 illuminates the observation target via the illumination optical system 30a.

The image pickup optical system 30b has an image pickup sensor 32 in which various types of light such as reflected light, scattered light, and fluorescence from an observation target are incident. The image sensor 32 forms an image of an observation target and outputs an image signal. The output image signal is transmitted to the processor device 16. The image sensor 32 includes a color image sensor such as an RGB color image sensor and a CMYG color image sensor, and a monochrome sensor without a color filter. As the image pickup sensor 32, a CCD (Charge Coupled Device) image pickup sensor, a CMOS (Complementary Metal-Oxide Semiconductor) image pickup sensor, or the like can be used.

Further, as shown in FIG. 5, a measuring scale 40 for measuring the insertion length of the insertion portion 12a into the body is provided on the outer peripheral surface of the insertion portion 12a of the endoscope 12. The measurement scale 40 is composed of points provided at a predetermined pitch (for example, in 1 cm increments) along the longitudinal direction of the insertion portion 12a. The measurement scale 40 is detected by a scale detection sensor 42 provided in the patient's mouth (in the case of an upper endoscope) or anus (in the case of a lower endoscope), and the scale detection sensor 42 is wired to the processor device 16. Alternatively, it is wirelessly connected, and the detection information of the scale detection sensor 42 is transmitted to the processor device 16. In FIG. 5, the scale detection sensor 42 is provided on the mouthpiece MP held by the patient in his mouth.

As shown in FIG. 4, the processor device 16 includes an image signal acquisition unit 50, an image processing unit 52, a display control unit 54, a still image storage control unit 55, a storage image storage unit 56, and a scope position information. It includes a calculation unit 57 and a scope operation information calculation unit 58. The image signal acquisition unit (corresponding to the “image acquisition unit” of the present invention) 50 acquires an image signal corresponding to each observation mode from the endoscope 12. The image processing unit 52 includes a normal light mode processing unit 60, a special light mode processing unit 62, and an inspection support mode processing unit 64. The normal light mode processing unit 60 performs normal light mode image processing such as color conversion processing, color enhancement processing, and structure enhancement processing on the image signal obtained in the normal light mode. As a result, a normal light image having a natural hue can be obtained. The normal optical image is input to the display control unit 54. When the still image acquisition instruction unit 13b is operated, the still image of the normal light image at that time is stored in the storage image storage unit 56 by the still image storage control unit 55.

The special light mode processing unit 62 performs special light mode image processing different from the normal light mode image processing on the image signal obtained in the special light mode to obtain a special light image. Regarding the image processing for the special light mode, the processing contents (color conversion processing, color enhancement processing, structure enhancement processing, etc.) are the same as those of the processed image for the normal light mode, but the processing conditions are different. In the special light image, the visibility of the lesion is higher than that of other parts. The special light image is input to the display control unit 54. When the still image acquisition instruction unit 13b is operated, the still image of the special light image at that time is stored in the storage image storage unit 56 by the still image storage control unit 55.

The inspection support mode processing unit 64 performs inspection support mode image processing on the image signal obtained in the inspection support mode to obtain an inspection image. The inspection image is input to the display control unit 54. In the image processing for the inspection support mode, when the still image acquisition instruction unit 13b is not operated, the inspection image is input to the display control unit 54 as it is. On the other hand, when the still image acquisition instruction unit 13b is operated, an inspection support image for confirming whether or not the image can be taken according to the guideline is generated. This inspection support image is input to the display control unit 54 together with the inspection image. Then, the user confirms the inspection support image, and further, the still image acquisition instruction unit 13b is operated, so that the still image of the inspection image at that time is stored by the still image storage control unit 55 by the storage image storage unit 56. Is remembered in. The details of the inspection support mode processing unit 64 will be described later.

The display control unit 54 controls to display the image input from the image processing unit 52 on the monitor 18. As a result, the image corresponding to the observation mode is displayed on the monitor 18. That is, in the normal light mode, the normal light image is displayed on the monitor 18, and in the special light mode, the special light image is displayed on the monitor 18. Further, in the inspection support mode, the inspection image is displayed on the monitor 18, and when the still image acquisition instruction unit 13b is operated, the inspection support image is displayed in addition to the inspection image.

The scope position information calculation unit 57 calculates the insertion length of the insertion unit 12a of the endoscope 12 based on the detection information of the scale detection sensor 42. Further, the scope position information calculation unit 57 calculates the scope position information indicating which of the inspection areas A1 to A13 the tip portion 12d of the endoscope 12 is located based on the insertion length of the insertion portion 12a. As shown in FIGS. 6 and 7, when the insertion length of the scope position information calculation unit 57 is between L0 and L1, the tip portion 12d is located in the inspection area A1 and the insertion lengths are L1 and L2. If it is in between, the tip 12d is located in the inspection area A2.

When the insertion length is between L2 and L3, the tip portion 12d is located in the inspection area A3. When the insertion length is between L3 and L4, the tip portion 12d is located in the inspection area A4. When the insertion length is between L4 and L5, the tip 12d is located in the inspection area A5. When the insertion length is between L5 and L6, the tip 12d is located in the inspection area A6. When the insertion length is between L6 and L7, the tip 12d is located in the inspection area A7. When the insertion length is between L7 and L8, the tip 12d is located in the inspection area A8. When the insertion length is between L8 and L9, the tip 12d is located in the inspection area A9. When the insertion length is between L9 and L10, the tip 12d is located in the inspection area A10.

When the insertion length is between L10 and L11, the tip portion 12d is located in the inspection area A11. When the insertion length is between L11 and L12, the tip 12d is located in the inspection area A12. When the insertion length is between L12 and L13, the tip 12d is located in the inspection area A13.

Considering that the positional relationship between the insertion length and the inspection area differs depending on the physique of the subject, the scope position information calculation unit 57 calibrates the positional relationship between the insertion length and the inspection area according to the physique difference. Is preferable. For example, the insertion length in the pylorus such as the inspection area A3 is compared with the predetermined insertion length of the pylorus for calibration, and the positional relationship between the insertion length and the inspection area is corrected based on the difference between them. May be good. Further, the distance between the pylorus portion such as the inspection area A3 and the cardia portion of the inspection area 10 is compared with the predetermined calibration pylorus and the distance between the cardia, and the insertion length is determined based on the difference. The positional relationship with the inspection area may be corrected.

The scope operation information calculation unit 58 calculates the scope operation information indicating the information regarding the operation of the endoscope 12. The scope operation information includes the angle angle or rotation amount of the tip portion 12d of the endoscope 12, the distance to the observation target (observation distance), the insertion length, the shape of the endoscope, and the like. The angle angle or the amount of rotation is calculated based on the operation information of the operation unit 12b such as the operation amount of the angle knob 13a. The observation distance is calculated based on the average value of the pixel values of the inspection image and the like. As described above, the insertion length is calculated by the scope position information calculation unit 57. The shape of the endoscope is preferably calculated using a magnetic sensor (not shown) provided at the tip portion 12d of the endoscope. In addition to the tip portion 12d, a plurality of magnetic sensors may be provided in the insertion portion 12a at equal intervals from the tip portion 12d.

As shown in FIG. 8, the inspection support mode processing unit 64 operates when the still image acquisition instruction unit 13b is operated in addition to the inspection support mode image processing unit 70 that performs inspection support mode image processing. The processing unit includes a blur determination unit 72, a defect region removal unit 73, a reference image selection unit 74, a similarity calculation unit 76, and an inspection support image generation unit 78. In the inspection support mode, after confirming whether the still image of the inspection image is an image in accordance with the guideline, the still image is stored in the storage image storage unit 56.

Therefore, the still image of the inspection image is photographed in two stages, a temporary image and a main image. The default is set to the state of temporary shooting, and when the still image acquisition instruction unit 13b is operated, temporary shooting is performed. After this provisional shooting, when the still image acquisition instruction unit 13b is further operated, the main shooting is performed, and the still image of the acquired inspection image is stored in the storage image storage unit 56. After being stored in the storage image storage unit 56, the state of temporary shooting is restored again.

As shown in FIG. 9, the blur determination unit 72 determines whether or not the still image of the temporarily photographed inspection image is out of focus or blurred. In order to determine the blur or blur, the blur determination unit 72 calculates a blur index value indicating the out-of-focus or blur of the still image of the temporarily photographed inspection image. As a method of calculating the blur index value, for example, since there is a relationship between blur and blur and the spatial frequency of the image, there is a method of calculating the blur index value based on the spatial frequency of the inspection image.

The blur determination unit 72 determines that the inspection image is blurred when the calculated blur index value exceeds the threshold value. Since it is difficult to accurately compare the inspection image in which the blurring occurs with the reference image, guidance for urging the user to retake the inspection image is displayed on the monitor 18. On the other hand, when the calculated blurring index value is equal to or less than the threshold value, the blurring determination unit 72 determines that the blurring does not affect the comparison with the reference image, and the comparison with the reference image is permitted. .. In this case, the inspection image is transmitted to the defect region removing unit 73.

The defect region removing unit 73 removes defect regions such as stains and halation on the observation target from the inspection image. For example, in the inspection image, a high pixel region having a pixel value of a certain value or more is recognized as a halation region, and the region of this portion is replaced with the pixel value of the adjacent region. The inspection image after removing the defective region is transmitted to the reference image selection unit 74.

The reference image selection unit 74 includes a reference image storage unit 74a that stores a plurality of predetermined reference images for each of the inspection areas A1 to A13. The reference image selection unit 74 performs a comparison process between the reference image stored in the reference image storage unit 74a and the inspection image for which blur determination has been determined or the defect region has been removed. The comparison process is performed by pattern matching or the like. Based on this comparison process, a reference image corresponding to the inspection image is selected from the reference image storage unit 74a. The selected reference image and inspection image are transmitted to the similarity calculation unit 76.

Further, the reference image storage unit 74a stores the reference image in the RAW data format so that the state of the reference image can be changed according to the inspection support mode image processing used in the inspection support mode processing unit 64. There is. Therefore, in the reference image selection unit 74, before performing the comparison processing with the inspection image, the reference image stored in the reference image storage unit 74a is subjected to the inspection support mode image processing (“specific image” of the present invention. (Corresponding to "processing") is performed, and then the reference image to which the image processing for the inspection support mode has been processed is compared with the inspection image. The reference image selection unit 74 may perform comparison processing between the image before the inspection support mode image processing and the reference image in the RAW data format. The RAW data is image data in a state where various image processing such as color tone enhancement processing (in this embodiment, image processing for normal light mode, image processing for special light mode, image processing for inspection support mode) has not been performed. Say.

The similarity calculation unit 76 calculates the similarity between the reference image and the inspection image based on the image data of the reference image and the image data of the inspection image. The calculated similarity with the reference image is transmitted to the inspection support image generation unit 78. As a method of calculating the similarity, for example, the noise contained in the image data of the reference image and the noise contained in the image data of the inspection image are compared, and the smaller the difference in the amount of the noise, the higher the similarity. .. In addition, the structure such as folds included in the image data of the reference image is compared with the structure included in the image data of the inspection image and landmarks such as cardia and pylorus, and the more similar the structure, the higher the similarity. It is said to be expensive. Further, when the insertion portion 12a of the endoscope 12 is reflected in the reference image, it is preferable to determine the degree of similarity depending on the presence or absence of the insertion portion 12a. In this case, if the inspection image includes the insertion portion 12a, the similarity is increased. Further, the similarity may be calculated by not only comparing the image data of the reference image and the image data of the inspection image but also adding the scope operation information. By adding the scope operation information in this way, the similarity can be calculated accurately.

Further, as a method of calculating the similarity, a method using a feature amount such as an image color can be considered, but since it is difficult to accurately calculate the similarity with only one feature amount, a plurality of feature amounts are used. Use the combined overall similarity so that the similarity can be calculated accurately. For example, when a color feature amount A, a structure feature amount B, and a landmark feature amount C are used as feature amounts, the total similarity is calculated by weighting and adding these feature amounts A to C. To do.

The above-mentioned total similarity may be used for selecting a reference image in the reference image selection unit 74. For example, as shown in FIG. 10, when there are 1 to 40 reference images, the similarity of the 40 reference images with the feature amounts A to C with the inspection image (in FIG. 10, the similarity is% (percentage)). ) Is calculated, and the total similarity is calculated based on the calculated similarity of the features A to C. Then, the reference image having the highest overall similarity is selected. In this case, the reference image having the highest overall similarity is the reference image No. Since it is 38, this reference image No. Select 38.

The inspection support image generation unit 78 generates an inspection support image based on the degree of similarity between the inspection image and the reference image. The generated inspection support image is transmitted to the display control unit 54 and displayed on the monitor 18. As shown in FIG. 11, in the inspection support image, the similarity with the reference image (for example, the similarity with the reference image of the pylorus is “80%”) is displayed together with the inspection image. As shown in FIG. 12, when the user confirms the similarity between the inspection image and the reference image and confirms that the similarity satisfies a specific condition (for example, a certain value or more), the inspection image becomes a guideline. It is determined that the image is in line with the image (reference equivalent image), and the main shooting is performed again by operating the still image acquisition instruction unit 13b. As a result, the inspection image is stored in the storage image storage unit 56. If the similarity with the reference image does not satisfy a specific condition such as a certain value or more, the display control unit 54 may display the warning display 80 on the monitor 18 as shown in FIG. Good.

On the other hand, when the user determines that the inspection image is not an image (standard equivalent image) in accordance with the guideline, such as when the similarity does not satisfy a specific condition, the still image acquisition instruction unit 13b is not operated for a certain period of time or longer. Alternatively, by operating the inspection image non-adopting button (not shown) provided on the endoscope 12 or the user interface 19, the storage of the inspection image in the storage image storage unit 56 is postponed. In this case, the display of the inspection support image is stopped, and the still image of the inspection image is taken again.

When retaking a still image of an inspection image, the endoscope 12 is operated so that the user can acquire the inspection image (inspection image whose similarity satisfies a specific condition) according to the guideline. It is preferable that the display control unit 54 displays the operation support information for support on the monitor 18. The operation support information includes information on how much the angle angle and insertion length should be adjusted in order to obtain an inspection image in accordance with the guideline, and how much the air pressure of the air supply gas to be blown to the observation target is. Contains information on what to do. For example, in the case of FIG. 14, the operation support information 82 regarding the angle angle and the insertion length is displayed. The operation support information may be automatically displayed on the monitor 18 when the similarity with the reference image does not satisfy a specific condition.

In the inspection support image, in addition to displaying the degree of similarity with the inspection image and the reference image, it is also possible to display in each inspection area whether the acquisition of the inspection image was successful or unsuccessful. As shown in FIG. 15, in the inspection support image, in addition to the similarity with the inspection image and the reference image, the inspection target map 90 which maps the inspection areas A1 to A13 is used as an image storage unit 56 for storing the inspection image. The display control unit 54 displays whether or not the image is saved in. In this inspection support image, it is displayed that the inspection images of the inspection areas A1 and A2 have been saved (“OK” is displayed in FIG. 15), and the still images of the inspection images of the inspection areas A3 and A4 have been acquired. It is displayed that the image has not been saved in the storage image storage unit 56 (“NG” is displayed in FIG. 15). Here, it is preferable that the OK region and the NG region are displayed in different colors (for example, the OK region is displayed in "blue" and the NG region is displayed in "yellow"). Information about the inspection area is acquired from the scope position information.

In addition, when the presence or absence of saving of the still image of the inspection image is displayed in the inspection support image, it becomes possible to detect the failure to take the inspection image. As shown in FIG. 16, when the inspection images of the inspection areas A1 to A10 and A12 are displayed as saved, and the inspection areas A11 between A10 and A12 are not displayed as saved, the inspection area A11 It can be seen that the photograph of (lower esophagus) was missed. In such a case, the insertion portion 12a of the endoscope 12 is pushed in again to take an image of the examination area A11 (lower esophagus).

As shown in FIG. 17, in the inspection support image, in addition to displaying the degree of similarity with the inspection image and the reference image, information regarding the scope operation information 84 and the scope position information 86 may be displayed. In this inspection support image, the insertion length and the angle angle are displayed as the scope operation information, and the reference insertion length and the reference angle angle when the reference image is obtained are also displayed. Further, as the scope position information, it is indicated in which inspection area the tip portion 12d of the endoscope is located (for example, the inspection area A3). Furthermore, in the examination support image, the shape of the insertion portion 12a of the endoscope 12 is used in the scope operation information to indicate the shape of the insertion portion 12a inserted in the stomach to be inspected. Has been done.

[Second Embodiment]
In the first embodiment, in the inspection support image, the similarity with the reference image is displayed together with the inspection image, but in the second embodiment, as shown in FIG. 18, the similarity with the reference image is displayed. Instead of, a reference image (for example, a reference image of the pylorus) may be displayed. In this case, the user operates the endoscope 12 so that the inspection image has the same composition as the reference image, and when the inspection image has the same structure, operates the still image acquisition instruction unit 13b to operate the inspection image. Acquires a still image of. Further, as shown in FIG. 19, in the inspection support image, in addition to the inspection image and the reference image, the scope operation information 84 and the scope position information 86 may be displayed together. In this case, the user operates the endoscope 12 while looking at the scope operation information to match the structure of the reference image.

Further, as shown in FIG. 20, in the inspection support image, in addition to the inspection image and the reference image, the inspection target map 90 is displayed, and in the inspection target map 90, the inspection image is stored in the storage image storage unit 56. The presence / absence may be displayed. Similar to the first embodiment, the display regarding the presence / absence of saving is OK for the inspection area where the inspection image has been saved in the storage image storage unit 56, and the still image of the inspection image is saved. NG is displayed for the inspection area for which the inspection image has not been saved in the image storage unit 56.

Further, as shown in FIG. 21, in the inspection support image, in addition to the inspection image and the reference image (for example, the reference image of the pylorus), the similarity with the reference image (similarity with the reference image of the pylorus is "80". % ") May be calculated and displayed. In this case, by using not only the reference image but also the similarity, it becomes possible to more reliably capture an image (reference equivalent image) according to the guideline. When the similarity is calculated in this way, in the second embodiment, various types of user support can be provided when the similarity does not satisfy a specific condition.

For example, as shown in FIG. 22, in the inspection support image, in addition to the inspection image, the reference image, and the similarity, a warning display 80 indicating that the similarity does not satisfy a specific condition may be displayed. Further, as shown in FIG. 23, the operation support information 82 necessary for taking the inspection image according to the guideline may be displayed in the inspection support image. In this way, by displaying the warning display 80 and the operation support information 82 in addition to the similarity with the reference image, it becomes possible to more reliably take an image (reference equivalent image) according to the guideline. Further, a function of incorporating a light emitting member such as an LED (Light Emitting Diode) in the angle knob 13a of the operation unit 12b and lighting it so that the direction of rotation or the like can be known may be added to support the operation of the endoscope.

In the above embodiment, the similarity with the reference image or the reference image is displayed so that the image can be taken according to the guideline in one endoscopy. Even in the subsequent follow-up, it is preferable to display a plurality of inspection images in the same inspection area but different acquisition timings on the monitor 18 so that an effective diagnosis can be made. In this case, in order to enable the follow-up observation after the lesion is found in the initial examination, the scope is used when the examination image is stored in the storage image storage unit 56 in the initial examination. It is preferable to store the position information or the scope operation information in association with each other.

As shown in FIG. 24, at the time of follow-up, the scope position information or the scope operation information at the position where the lesion is found is set in advance, and the tip of the endoscope is set at the position where the lesion is found. When the part 12d is reached, the previous inspection image is displayed together with the inspection image. In addition to the previous examination image, the size of the lesion (polyp) at the time of this follow-up and the size of the lesion (polyp) at the time of the previous follow-up are displayed. Further, a reference image may be displayed.

[Third Embodiment]
In the first and second embodiments, when a still image of the inspection image is acquired by a temporary shooting by the user, the similarity with the reference image or the reference image is displayed, and after confirming the similarity or the reference image, The main shooting is performed and the still image of the inspection image is stored in the storage image storage unit. Instead, in the third embodiment, the inspection image is acquired at the timing of displaying the moving image of the inspection image. When the obtained inspection imaging conditions satisfy the reference imaging conditions for obtaining a reference image, provisional imaging is automatically performed to acquire a still image of the inspection image.

In the endoscope system 10 of the third embodiment, instead of the inspection support mode, an automatic imaging mode for automatically acquiring a still image of an inspection image is provided when the imaging conditions at the time of inspection satisfy the standard imaging conditions. Has been done. Therefore, as shown in FIG. 25, in the processor device 16 of the endoscope system 10, the processing unit 100 for the automatic photographing mode is provided in place of the processing unit 64 for the inspection support mode of the first and second embodiments. ing. The switching to the automatic shooting mode is performed by the mode switching unit 13c.

In addition to the reference image selection unit 74, the similarity calculation unit 76, and the inspection support image generation unit 78 shown in the first and second embodiments, the automatic shooting mode processing unit 100 includes an automatic shooting mode image processing unit 102. A shooting condition acquisition unit 104 at the time of inspection, a shooting condition determination unit 106, and a still image storage request signal output unit 108 are provided. In the third embodiment, since the inspection image and the reference image are not compared, the blur determination unit 72 and the defect area removal unit 73 are not provided, but in addition to the comparison between the inspection imaging condition and the reference imaging condition. When comparing the inspection image and the reference image, it is preferable that the blur determination unit 72 and the defect region removal unit 73 are also performed together.

The image processing unit 102 for the automatic shooting mode performs image processing for the automatic shooting mode on the image signal obtained in the automatic shooting mode. The image processing for the automatic shooting mode is the same as the image processing for the inspection support mode of the first and second embodiments, and the inspection image can be obtained by performing the image processing for the automatic shooting mode. The inspection image is input to the display control unit 54 and displayed as a moving image on the monitor 18.

The inspection imaging condition acquisition unit 104 acquires the imaging conditions when the inspection image is acquired as the inspection imaging conditions. This inspection imaging condition includes at least one of the scope position information and the scope operation information when the inspection image is acquired. Therefore, the inspection shooting condition acquisition unit 104 accesses the scope position information calculation unit 57 and the scope operation information calculation unit 58 every time the inspection image for a predetermined frame is acquired, and acquires the scope position information or the scope operation information. ..

The imaging condition determination unit 106 determines whether or not the imaging conditions at the time of inspection satisfy the reference imaging conditions for obtaining a reference image. The shooting condition determination unit 106 includes a reference shooting condition storage unit 106a that stores reference shooting conditions for obtaining a reference image defined in the guideline. For example, when there are 1 to 40 reference images, as shown in FIG. 26, the reference image No. As a reference photographing condition for obtaining 1, the reference image No. The reference shooting condition of 1 is stored, and the other reference image No. 2-Reference image No. As for 40, the reference image No. 40 is also used as a reference shooting condition for obtaining these reference images. 2 standard shooting conditions-reference image No. Forty reference shooting conditions are stored.

The reference shooting condition includes at least one of scope position information and scope operation information for obtaining a reference image. Further, the fact that the imaging conditions at the time of inspection satisfy the standard imaging conditions means, for example, the following. When the shooting condition is the insertion length, the insertion length Lx at the time of acquiring the inspection image is included in a certain range Ly1 to Ly2 (Ly2> Ly1) including the insertion length Ly for obtaining a predetermined reference image. In addition, it is said that the shooting conditions at the time of inspection meet the standard shooting conditions.

When the still image storage request signal output unit 108 determines in the imaging condition determination unit 106 that the inspection imaging condition satisfies any of the plurality of reference imaging conditions stored in the reference imaging condition storage unit 106a, the still image storage request signal output unit 108 determines. As shown in FIG. 27, a still image storage request signal requesting the display control unit 54 or the still image storage control unit 55 to save the inspection image in the still image storage unit 56 is output. When the imaging condition determination unit 106 determines that the imaging conditions at the time of inspection do not satisfy any of the standard imaging conditions, the user sets the endoscope 12 so as to be located at the inspection site defined in the guideline. Operate to reacquire the imaging conditions at the time of inspection.

Upon receiving the still image storage request signal, the display control unit 54 displays on the monitor 18 a confirmation message 120 for confirming whether or not to save the inspection image as a still image together with the inspection image, as shown in FIG. Let me. When the still image storage request signal is output, the reference image selection unit 74 selects a reference image corresponding to the reference imaging condition satisfying the inspection imaging condition from the reference image storage unit 74a. The selected reference image is displayed on the monitor 18 by the display control unit 54 (in FIG. 28, the “reference image of the pylorus portion” is displayed). In the reference image storage unit 74a of the third embodiment, the reference image is stored in association with the reference imaging condition so that the reference image can be selected from the reference imaging conditions.

Further, the similarity calculation unit 76 calculates the similarity between the reference image selected by the reference image selection unit 74 at the time of outputting the still image storage request signal and the inspection image. The calculated similarity is displayed on the monitor 18 by the display control unit 54 (in FIG. 28, “similarity with the reference image of the pylorus (80%)” is displayed).

As shown in FIG. 29, the still image storage control unit 55 receives the still image storage request signal, and the user sees the still image of the inspection image displayed on the monitor 18 as an image (reference equivalent image) according to the guideline. ), And when the operation of the still image acquisition instruction unit 13b is performed, the control is performed to store the still image of the inspection image in the storage image storage unit 56. When the still image of the inspection image is saved, as shown in FIG. 30, the display control unit 54 displays on the monitor 18 with a save completion message 122 indicating that the still image has been saved. In addition, the display control unit 54 displays the inspection area in which the still image is saved on the inspection target map 90. In the inspection target map 90, OK is displayed in the inspection area where the still image is saved in the inspection areas A1 to A13.

On the other hand, although the still image storage control unit 55 receives the still image storage request signal, the user can see that the still image of the inspection image displayed on the monitor 18 is the still image of the inspection image according to the guideline (reference equivalent image). ), And if the still image acquisition instruction unit 13b is not operated within a certain period of time after receiving the still image storage request signal, the inspection image is saved in the image storage unit 56 for storage. I can't. Further, even when the inspection image non-adoption button (not shown) provided on the endoscope 12 or the user interface 19 is operated even though the still image storage request signal is received, the inspection image storage image storage unit 56 It is not saved to. If the still image of the automatically acquired inspection image is not saved, the mode may be automatically switched to the manual shooting mode in which the still image of the inspection image is manually acquired. In this case, it is preferable to display on the monitor 18 that the manual shooting mode has been switched.

As described above, when the inspection image is not saved in the storage image storage unit 56 for saving, the display of the confirmation message 120 for confirming whether to save as a still image is also stopped. Instead of this confirmation message 120, as shown in FIG. 31, the monitor 18 displays the operation support information 82 necessary for taking an image (reference equivalent image) according to the guideline. Then, the user operates the endoscope 12 while checking the operation support information 82 on the monitor 18 to reacquire the imaging conditions at the time of examination.

[Fourth Embodiment]
In the third embodiment, when the inspection imaging condition satisfies the standard imaging condition, the still image of the inspection image is automatically acquired, but in the fourth embodiment, the inspection imaging condition is the reference imaging. When the condition is satisfied, the user is assisted in acquiring (shooting) the still image of the inspection image by notifying that it is the acquisition timing of the still image of the inspection image, that is, notifying that it is the shutter timing.

In the endoscope system 10 of the fourth embodiment, instead of the inspection support mode of the first and second embodiments, when the imaging condition at the time of inspection satisfies the standard imaging condition, the acquisition timing of the still image of the inspection image is notified. A shooting assist mode is provided. Therefore, as shown in FIG. 32, in the processor device 16 of the endoscope system 10, a photographing assist mode processing unit 200 is provided in place of the inspection support mode processing unit 64 of the first and second embodiments. ing. The mode switching unit 13c switches to the shooting assist mode. Further, in the fourth embodiment, in addition to the shooting assist mode, the automatic shooting mode of the third embodiment may be provided to switch the mode between the shooting assist mode and the automatic shooting mode.

The imaging assist mode processing unit 200 includes the reference image selection unit 74, the similarity calculation unit 76, and the inspection support image generation unit 78 shown in the first and second embodiments, and the inspection-time imaging shown in the third embodiment. In addition to the condition acquisition unit 104 and the shooting condition determination unit 106, an image processing unit 202 for shooting assist mode and a still image acquisition timing notification unit 204 are provided. In the fourth embodiment, since the inspection image and the reference image are not compared, the blur determination unit 72 and the defect area removal unit 73 are not provided, but in addition to the comparison between the inspection imaging condition and the reference imaging condition. When comparing the inspection image and the reference image, it is preferable to perform the processing by the blur determination unit 72 or the defect region removal unit 73 together.

The image processing unit 202 for the shooting assist mode performs image processing for the shooting assist mode on the image signal obtained in the shooting assist mode. The image processing for the shooting assist mode is the same as the image processing for the inspection support mode of the first and second embodiments, and the inspection image can be obtained by performing the image processing for the shooting assist mode. The inspection image is input to the display control unit 54 and displayed as a moving image on the monitor 18.

Also in the fourth embodiment, as in the third embodiment, the imaging conditions when the inspection image is acquired are acquired by the inspection imaging condition acquisition unit 104 as the inspection imaging conditions. Then, the imaging condition determination unit 106 determines whether or not the acquired imaging conditions at the time of inspection satisfy the reference imaging conditions. Since the imaging conditions at the time of inspection include at least one of the scope position information and the scope operation information when the inspection image is acquired, the scope position information calculation unit 57 and the scope position information calculation unit 57 each time an inspection image for a predetermined frame is obtained. Access the scope operation information calculation unit 58 to acquire the scope position information or the scope operation information.

When the shooting condition determination unit 106 determines that the inspection shooting condition satisfies the reference shooting condition, the still image acquisition timing notification unit 204 determines that the still image acquisition timing of the inspection image is as shown in FIG. 33. Is notified. When the imaging condition determination unit 106 determines that the imaging conditions at the time of inspection do not satisfy any of the standard imaging conditions, the user sets the endoscope 12 so as to be located at the inspection site defined in the guideline. Operate to reacquire the imaging conditions at the time of inspection.

As shown in FIG. 34, the still image acquisition timing, that is, the shutter timing notification 210 is displayed on the monitor 18 via the display control unit 54. Further, when the acquisition timing of the still image is notified, the reference image selection unit 74 selects the reference image corresponding to the reference imaging condition satisfying the imaging condition at the time of inspection, as in the third embodiment. Then, the selected reference image (“reference image of the pylorus portion” in FIG. 34) is displayed on the monitor 18. Further, the similarity calculation unit 76 calculates the similarity between the reference image and the inspection image selected by the reference image selection unit 74. The calculated similarity (“similarity (80%) with the reference image of the pylorus portion” in FIG. 34) is also displayed on the monitor 18. The still image acquisition instruction unit 13b in the operation unit 12b is configured by a button with a built-in LED (Light Emitting Diode) or the like, and the LED built in the still image acquisition instruction unit 13b is turned on at the time of the still image acquisition timing. , The acquisition timing of the still image may be notified.

As shown in FIG. 35, the still image storage control unit 55 notifies the acquisition timing of the still image, and the user follows the guideline that the still image of the inspection image displayed on the monitor 18 is the still image of the inspection image. When it is determined that the image (reference equivalent image) is used and the still image acquisition instruction unit 13b is operated, the still image of the inspection image is stored in the storage image storage unit 56 as in the third embodiment. Control to memorize. When the still image of the inspection image is saved, it is preferable to display the inspection area where the still image is saved on the inspection target map 90 as in the third embodiment (see FIG. 30). ).

On the other hand, although the still image storage control unit 55 is notified of the acquisition timing of the still image, the user can use the still image of the inspection image displayed on the monitor 18 as the still image of the inspection image according to the guideline (corresponding to the reference). When it is determined that the image is not) and the still image acquisition instruction unit 13b is not operated within a certain period of time from the notification (in addition, when the inspection image non-adoption button shown in the third embodiment is operated). Is not stored in the image storage unit 56 for storing the still image of the inspection image. In this case, the user operates the endoscope 12 so as to be located at the inspection site defined in the guideline, and reacquires the imaging conditions at the time of inspection. When the still image is not saved, it is preferable to display the operation support information 82 necessary for taking an image (reference equivalent image) according to the guideline on the monitor 18 as in the third embodiment (as in the third embodiment). See FIG. 31).

In the above embodiment, the hardware structure of the processing unit that executes various processes such as the image processing unit 52 is various processors as shown below. For various processors, the circuit configuration is changed after manufacturing the CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), which is a general-purpose processor that executes software (program) and functions as various processing units. It includes a programmable logic device (PLD), which is a possible processor, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for performing various processes, and the like.

One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). May be done. Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software. There is a form in which this processor functions as a plurality of processing units. Secondly, as typified by System On Chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. is there. As described above, the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.

Further, the hardware structure of these various processors is, more specifically, an electric circuit in the form of combining circuit elements such as semiconductor elements.

10 Endoscope system 12 Endoscope 12a Insertion part 12b Operation part 12c Curved part 12d Tip part 13a Angle knob 13b Still image acquisition instruction part 13c Mode switching part 13d Zoom operation part 14 Light source device 16 Processor device 18 Monitor 19 User interface 20 Light source 22 Light source control 24 Light guide 30a Illumination optical system 30b Imaging optical system 32 Imaging sensor 42 Detection sensor 50 Image signal acquisition unit 52 Image processing unit 54 Display control unit 55 Still image storage control unit 56 Storage image storage unit 57 Scope Position information calculation unit 58 Scope operation information calculation unit 60 Normal light mode processing unit 62 Special light mode processing unit 64 Inspection support mode processing unit 70 Inspection support mode image processing unit 72 Blurring judgment unit 73 Defect area removal unit 74 Reference Image selection unit 74a Reference image storage unit 76 Similarity calculation unit 78 Inspection support image generation unit 80 Warning display 82 Operation support information 84 Scope operation information 86 Scope position information 90 Inspection target map 100 Processing unit for automatic shooting mode 102 For automatic shooting mode Image processing unit 104 Shooting condition acquisition unit during inspection 106 Shooting condition acquisition unit 108 Reference shooting condition storage unit 108 Still image save request signal output unit 120 Confirmation message 122 Save completion message 200 Shooting assist mode processing unit 202 Shooting assist mode image processing Unit 204 Still image acquisition timing notification unit 210 Shutter timing notification

Claims (13)

A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
From the image data of the reference image selected by the reference image selection unit and the image data of the inspection image, the feature amount of the color and the feature of the structure related to the reference image selected by the reference image selection unit and the inspection image. The amount and / or the feature amount of the landmark is calculated, and the feature amount similarity is calculated for the feature amount of the color related to the reference image and the inspection image, the feature amount of the structure and / or the feature amount of the landmark, respectively. Then, a similarity calculation unit that performs weighting addition to each of the feature quantity similarity and calculates the total similarity as a combination of the plurality of feature quantity similarity, and the similarity calculation unit.
Wherein when the degree of similarity is not Is was satisfy the specific condition, the processor unit and a display control unit for warning display to the display unit. A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. ,
A display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition is provided.
A processor device that increases the degree of similarity when the reference image includes an insertion portion of an endoscope and the inspection image includes the insertion portion. A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. ,
A display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition is provided.
The similarity calculation unit is a reference selected by the reference image selection unit from the image data of the reference image selected by the reference image selection unit, the image data of the inspection image, and the scope operation information related to the operation of the endoscope. A processor device that calculates the similarity between an image and the inspection image. A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. ,
A display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition is provided.
The similarity calculation unit is a reference selected by the reference image selection unit from the image data of the reference image selected by the reference image selection unit, the image data of the inspection image, and the scope operation information related to the operation of the endoscope. The similarity between the image and the inspection image is calculated, and
The scope operation information includes at least one of the angle angle or rotation amount of the endoscope, the distance to the observation target, the insertion length of the insertion portion of the endoscope, and the shape of the insertion portion of the endoscope. A processor device that includes one. A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. ,
A display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition is provided.
The display control unit is a processor device that displays operation support information related to the operation of the endoscope so that the similarity satisfies the specific condition. A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. ,
A display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition is provided.
The reference image storage unit stores the reference image in the RAW data format.
The similarity calculation unit performs specific image processing applied to the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image to perform the reference image in the RAW data format. A processor device that calculates the degree of similarity between the reference image and the inspection image selected by the reference image selection unit. A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. ,
A display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition.
A defect region removing portion for removing a defective region from the inspection image is provided.
The similarity calculation unit removes the reference image selected by the reference image selection unit and the defect region from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. A processor device that calculates the degree of similarity with an inspection image. A reference image storage unit that stores a plurality of predetermined reference images for each inspection area,
An image acquisition unit that captures the inspection area and acquires an inspection image in order to obtain a reference equivalent image corresponding to the reference image.
A reference image selection unit that selects a reference image corresponding to the inspection image from the reference image storage unit,
A similarity calculation unit that calculates the similarity between the reference image selected by the reference image selection unit and the inspection image from the image data of the reference image selected by the reference image selection unit and the image data of the inspection image. ,
A display control unit that displays a warning to the display unit when the similarity does not satisfy a specific condition is provided.
When the display control unit stores the inspection image in the specific inspection area in the storage image storage unit, the display control unit displays on the inspection target map that the inspection image in the specific inspection area has been saved. apparatus. The claim that the similarity calculation unit calculates the similarity by comparing the noise, structure or landmark contained in the image data of the reference image with the noise, structure or landmark contained in the image data of the inspection image. The processor device according to any one of 1 to 8 . The processor device according to any one of claims 1 to 9 , wherein the specific condition is a certain degree or more of the similarity. The processor device according to any one of claims 1 to 10 , wherein the inspection image is an image obtained when the still image acquisition instruction unit is operated. The processor device according to any one of claims 1 to 11 .
An endoscope system including a display unit that displays the degree of similarity. The image acquisition unit includes a step of imaging the inspection area and acquiring an inspection image in order to obtain a reference equivalent image corresponding to a predetermined reference image for each inspection area.
A step in which the reference image selection unit selects a reference image corresponding to the inspection image from the reference image storage unit that stores the reference image.
The similarity calculation unit is based on the image data of the reference image selected by the reference image selection unit and the image data of the inspection image, and the color characteristics of the reference image and the inspection image selected by the reference image selection unit. The amount, the feature amount of the structure and / or the feature amount of the landmark are calculated, and the feature amount of the color related to the reference image and the inspection image, the feature amount of the structure and / or the feature amount of the landmark, respectively. A step of calculating the feature amount similarity, weighting and adding each of the feature amount similarity, and calculating the total similarity obtained by combining a plurality of the feature amount similarity as the similarity .
A method of operating a processor device, wherein the display control unit includes a step of displaying a warning on the display unit when the similarity does not satisfy a specific condition.

Images