JP2018007837A - Image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of reducing a circuit scale or processing time required when simultaneously displaying a plurality of images acquired by imaging a subject.SOLUTION: An image processing device includes: a parallax image generation unit for generating a plurality of parallax images corresponding to a plurality of optical images formed according to the light from a subject; and a display control unit for performing an operation for displaying an image for display according to the plurality of parallax images. It also includes a setting unit which sets one parallax image out of the plurality of parallax images as a main parallax image, and which sets one or more parallax images other than the main parallax image out of the plurality of parallax images as sub parallax images. When the main parallax image and the sub parallax images are simultaneously displayed as the image for display, an operation is performed for reflecting a parameter acquired from the main parallax image to the sub parallax images included in the image for display.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus capable of simultaneously displaying a plurality of images obtained by imaging a subject.

  A technique related to processing for simultaneously displaying a plurality of images obtained by imaging a subject with an imaging device such as an endoscope is known.

  Specifically, for example, in Patent Document 1, in an endoscope apparatus, two subject images obtained by an endoscope equipped with a binocular adapter in response to a tilting operation of a predetermined switch provided in a remote controller. A configuration for performing a process of switching between binocular display that displays the image on two screens and maximum display (full screen display) that displays one of the two subject images on one screen is disclosed. .

  Here, an image obtained by imaging a subject with an imaging device such as an endoscope is generally displayed on a display device after one or more image processes are performed. Therefore, for example, when simultaneously displaying a plurality of images obtained by imaging a subject with an imaging device such as an endoscope, the image processing is performed on each of the plurality of images. It is considered that the circuit scale or processing time tends to increase.

  However, Patent Document 1 does not particularly mention image processing performed on two subject images that are displayed simultaneously during binocular display. Therefore, according to the configuration disclosed in Patent Document 1, the circuit scale or the processing time is unnecessarily increased because different image processing is performed on each of the two subject images displayed simultaneously during binocular display. There is a problem that there is a risk of being lost.

  The present invention has been made in view of the above-described circumstances, and provides an image processing apparatus capable of reducing the circuit scale or processing time required when simultaneously displaying a plurality of images obtained by imaging a subject. It is an object.

  An image processing apparatus according to an aspect of the present invention includes a parallax image generation unit configured to generate a plurality of parallax images corresponding to a plurality of optical images formed according to light from a subject, and the plurality of parallaxes And a display control unit configured to perform an operation for displaying a display image according to an image, wherein one of the plurality of parallax images is converted into a main parallax image. A setting unit configured to set one or more parallax images other than the main parallax image among the plurality of parallax images as a sub parallax image. When the image and the sub parallax image are simultaneously displayed as the display image, the parameter obtained from the main parallax image is reflected in the sub parallax image included in the display image. Wherein the operation of the is performed.

  According to the image processing apparatus of the present invention, it is possible to reduce the circuit scale or processing time required when simultaneously displaying a plurality of images obtained by imaging a subject.

The figure which shows the structure of the principal part of the stereoscopic endoscope system containing the image processing apparatus which concerns on embodiment. The schematic diagram for demonstrating an example of the layout of the front end surface of an endoscope. 6 is a flowchart for explaining an example of an operation performed in the image processing apparatus according to the embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  1 to 3 relate to an embodiment of the present invention.

  As shown in FIG. 1, the stereoscopic endoscope system 1 includes an endoscope 2, a light source device 3, a processor 4, a display device 5, and an input device 6. FIG. 1 is a diagram illustrating a configuration of a main part of a stereoscopic endoscope system including an image processing apparatus according to an embodiment.

  The endoscope 2 includes an elongated insertion portion 2A that is inserted into a body cavity of a subject. Further, the endoscope 2 is configured as a direct-viewing type flexible mirror in which a visual field direction is set from the distal end portion of the flexible insertion portion 2A to the front side. The endoscope 2 is configured to be detachably connected to the light source device 3 and the processor 4 via a scope connector (not shown). In addition, a signal processing unit 24 and a nonvolatile memory 25 are provided inside the endoscope 2.

  A light guide 7 configured to transmit illumination light supplied from the light source device 3 to the distal end portion of the insertion portion 2A is inserted into the insertion portion 2A. In addition, an illumination optical system 21 that irradiates the subject with illumination light emitted through the light guide 7 and return light (reflected light) generated from the subject illuminated by the illumination light are provided at the distal end of the insertion portion 2A. And an imaging unit 22 that outputs an imaging signal. In addition, a treatment instrument channel 23 through which an elongated treatment instrument 8 used for treatment of the affected area existing in the body cavity of the subject can be inserted is provided inside the insertion section 2A. The treatment tool 8 includes, for example, a treatment portion 81 formed by an electrode or the like for applying electric energy or heat energy to the affected part in the body cavity at the distal end, and the energy output from the treatment portion 81 An operation unit 82 capable of performing an operation related to an output level change or the like is provided at the base end portion. That is, the treatment tool 8 is configured to perform a predetermined treatment on the affected part in the body cavity of the subject.

  The illumination optical system 21 is configured to irradiate the subject with illumination light supplied from the light source device 3 from the distal end surface 2B of the distal end portion of the insertion portion 2A. Specifically, the illumination optical system 21 has a lens 21A that receives illumination light emitted through the light guide 7 and illumination light emitted through the lens 21A from the front end surface 2B to the front side of the front end surface 2B. And an illumination window 21B that is an optical member that irradiates a subject to be positioned.

  The imaging unit 22 is configured to have a visual field direction on the front side of the distal end surface 2B. The imaging unit 22 is configured to form two optical images corresponding to incident light incident from the distal end surface 2B and to capture the two formed optical images, respectively. Specifically, the imaging unit 22 includes an observation window 22A where incident light such as return light from a subject illuminated by illumination light emitted through the illumination optical system 21 is incident, and incident incident from the observation window 22A. The optical system includes an objective optical system 22L that forms an optical image corresponding to light, an objective optical system 22R that forms an optical image corresponding to the incident light, and an image sensor 22B.

  The image sensor 22B includes, for example, a CCD or a CMOS. Further, the image sensor 22B captures two optical images, an optical image formed by the objective optical system 22L and an optical image formed by the objective optical system 22R, respectively, and according to the two captured optical images. The image pickup signal is generated, and the generated image pickup signal is output to the signal processing unit 24.

  Note that the imaging unit 22 is not limited to having the configuration as described above, and may be configured to include, for example, two cameras arranged at positions adjacent to each other at the distal end of the insertion unit 2A. The incident light incident from the observation window 22A may be divided into pupils by the field stop and imaged by the image sensor 22B, or the incident light incident from the observation window 22A may be captured for each pixel of the image sensor 22B. Alternatively, the image may be divided into pupils and imaged.

  The treatment instrument channel 23 includes an insertion port 23A that is an opening provided at the proximal end portion of the insertion portion 2A, and a projection port 23B that is an opening provided at the distal end surface 2B. In addition, the treatment instrument channel 23 is formed as a conduit that can project the distal end portion of the treatment instrument 8 inserted from the insertion opening 23A from the projection opening 23B.

  On the other hand, the front end surface 2B is provided with an illumination window 21B, an observation window 22A, and a protruding port 23B, for example, having a layout as shown in FIG. 2, for convenience of illustration and description, the light incident surfaces of the objective optical systems 22L and 22R arranged at the rear stage of the observation window 22A are used as the illumination window 21B, the observation window 22A, and the protrusion 23B. Similarly to the above, each is indicated by a solid line. FIG. 2 is a schematic diagram for explaining an example of the layout of the distal end surface of the endoscope.

  The protrusion 23B is formed at a position on the distal end surface 2B in FIG. 2 such that the distance DR to the optical axis OR of the objective optical system 22R is smaller than the distance DL to the optical axis OL of the objective optical system 22L. Yes. That is, the protrusion 23B is formed at a position closer to the objective optical system 22R side on the distal end surface 2B in FIG. In other words, the protrusion 23B is formed at a position on the right side with respect to the visual field direction of the imaging unit 22 on the distal end surface 2B of FIG.

  The signal processing unit 24 includes, for example, a signal processing circuit. In addition, the signal processing unit 24 generates a digital imaging signal by performing predetermined signal processing such as noise reduction processing and A / D conversion processing on the analog imaging signal output from the image sensor 22B, for example. The generated digital imaging signal is output to the processor 4.

  Information unique to each endoscope 2 is stored in the memory 25. Specifically, in the memory 25, for example, one of the two objective optical systems 22L and 22R corresponding to the model of the endoscope 2 is preset as a main optical system. Endoscopic information indicating that the information is present is stored. The endoscope information stored in the memory 25 is read out by a setting unit 42 (described later) of the processor 4 when the endoscope 2 and the processor 4 are connected and the power of the processor 4 is turned on. It is.

  The light source device 3 includes, for example, a white light source and is configured to supply white light emitted from the white light source to the light guide 7 as illumination light.

  The processor 4 has a function as an image processing apparatus. Specifically, the processor 4 includes a parallax image generation unit 41, a setting unit 42, a parallax image processing unit 43, a display control unit 44, and a light control unit 45. According to the present embodiment, for example, each part of the processor 4 may be configured as an individual electronic circuit, or configured as a circuit block in an integrated circuit such as an FPGA (Field Programmable Gate Array). Also good.

  The parallax image generation unit 41, based on the imaging signal output from the signal processing unit 24, the left-eye parallax image corresponding to the optical image formed by the objective optical system 22L and the optical image formed by the objective optical system 22R And a parallax image for the right eye corresponding to each of the two, and the generated two parallax images are output to the parallax image processing unit 43 and the dimming unit 45. That is, since the objective optical system 22L and the objective optical system 22R that receive incident light from the same subject are arranged side by side in the insertion portion 2A, an image corresponding to the optical image formed by the objective optical system 22L and the objective The image corresponding to the optical image formed by the optical system 22R is an image having parallax.

  The setting unit 42, based on the endoscope information read from the memory 25, out of the two parallax images generated by the parallax image generation unit 41, the main optical included in the endoscope information read from the memory 25. One parallax image corresponding to the optical image formed by the system is set as a main parallax image, and the other parallax image different from the one parallax image is set as a sub parallax image. . Further, the setting unit 42 outputs setting information indicating the main parallax image and the sub parallax image set as described above to the parallax image processing unit 43, the display control unit 44, and the dimming unit 45, respectively. It is configured.

  The parallax image processing unit 43 is configured to identify the main parallax image and the sub parallax image in the two parallax images output from the parallax image generation unit 41 based on the setting information output from the setting unit 42. Yes.

  The parallax image processing unit 43 acquires a processing parameter corresponding to the main parallax image, performs predetermined image processing using the acquired processing parameter on the main parallax image, and outputs the processed parallax image to the display control unit 44. Is configured to do. In addition, the parallax image processing unit 43 performs the predetermined image processing on the sub parallax image using the same processing parameters obtained when the predetermined image processing is performed on the main parallax image. Are output to the display control unit 44.

  The display control unit 44 is configured to identify the main parallax image and the sub parallax image in the two parallax images output from the parallax image processing unit 43 based on the setting information output from the setting unit 42. .

  The display control unit 44 includes a memory 44A that can store N (2 ≦ N) main parallax images and sub parallax images. In addition, the display control unit 44 combines the two parallax images simultaneously output from the parallax image processing unit 43 as a set of parallax images and stores the combined parallax images in the memory 44A. It is configured to update sequentially. That is, according to such a configuration and operation of the display control unit 44, the latest N sets of parallax images are stored in the memory 44A in time series.

  The display control unit 44 calculates the blur amount in the blur evaluation area set for each of the N main parallax images stored in the memory 44A during a period when the freeze switch (not shown) of the input device 6 is turned off. Processing for calculation is performed. Then, the display control unit 44 is combined with one main parallax image PMF having the smallest blur amount among the N main parallax images stored in the memory 44A and the one main parallax image PMF. In this state, each sub parallax image PSF stored in the memory 44A is specified.

  Note that the blur evaluation area described above may be set, for example, in the entire area of the main parallax image as long as the blur amount having an accuracy capable of specifying the main parallax image PMF is calculated. The blur evaluation area described above is preferably set to an area including the central portion of N main parallax images stored in the memory 44A.

  On the other hand, the display control unit 44 may perform a process of calculating a color misregistration amount instead of the process of calculating the blur amount as described above. Specifically, for example, the display control unit 44 calculates the color misregistration amount in the color misregistration evaluation area set for each of the N main parallax images stored in the memory 44A, and further, the N main Of these parallax images, one having the smallest color misregistration amount may be specified as one main parallax image PMF.

  Note that the above-described color misregistration evaluation area may be set, for example, in the entire area of the main parallax image as long as the color misregistration amount having an accuracy capable of specifying the main parallax image PMF is calculated. In addition, the color misregistration evaluation area described above is preferably set to an area including the central portion of the N main parallax images stored in the memory 44A.

  The display control unit 44 is configured to perform an operation for displaying the main parallax image and the sub parallax image on the display device 5 in a display mode according to an instruction or operation performed on the input device 6. .

  Specifically, for example, when the display control unit 44 detects that an instruction for setting to 3D display has been performed in a display changeover switch (not shown) of the input device 6, the display control unit 44 performs two main and sub An operation for simultaneously displaying the parallax image on the display device 5 as an observation image for stereoscopic viewing is performed. The display control unit 44 displays the main parallax image on the display device 5 as an observation image, for example, when it is detected that an instruction for setting to 2D display has been performed on the display changeover switch of the input device 6. It is comprised so that the operation | movement for making it may perform. For example, when the display control unit 44 detects that a freeze switch (not shown) of the input device 6 is turned on during 3D display, the display control unit 44 uses the main parallax image PMF and the sub parallax image PSF for stereoscopic viewing. An operation for simultaneously displaying the image on the display device 5 as a freeze image is performed. For example, when the display control unit 44 detects that a freeze switch (not shown) of the input device 6 is turned on during 2D display, the display control unit 44 displays the main parallax image PMF on the display device 5 as a freeze image. It is comprised so that operation | movement may be performed.

  The dimming unit 45 is configured to identify the main parallax image and the sub parallax image in the two parallax images output from the parallax image generation unit 41 based on the setting information output from the setting unit 42. .

  The light control unit 45 is configured to perform control for the light source device 3 to adjust the amount of illumination light according to the brightness of the main parallax image.

  The display device 5 includes, for example, an LCD (Liquid Crystal Display) and is configured to display an image according to the operation of the display control unit 44 on the display screen.

  The input device 6 includes a user interface such as a keyboard, a touch panel, and / or a foot switch. In addition, the input device 6 can be instructed to the processor 4 to set the display mode of the observation image displayed on the display device 5 to either 3D display or 2D display in accordance with a user operation. A display changeover switch is provided. Further, the input device 6 is provided with a freeze switch capable of performing an operation for switching the display of the freeze image on the display device 5 on or off in accordance with a user operation.

  Next, the operation of this embodiment will be described with reference to FIG. In the following description, a case where endoscope information indicating that the objective optical system 22L is set as the main optical system is stored in the memory 25 will be described as an example. In the following, a case where white balance processing is performed in the parallax image processing unit 43 will be described as an example. FIG. 3 is a flowchart for explaining an example of an operation performed in the image processing apparatus according to the embodiment.

  The user connects each part of the stereoscopic endoscope system 1 and turns on the power. And according to such user operation, the illumination light supplied from the light source device 3 is irradiated to the subject through the illumination optical system 21, and the subject is imaged by the imaging unit 22, and formed by the objective optical system 22L. The parallax image PL for the left eye corresponding to the optical image and the parallax image PR for the right eye corresponding to the optical image formed by the objective optical system 22R are generated by the parallax image generation unit 41, and the generated parallax image PL and PR are output to the parallax image processing unit 43 and the light control unit 45.

  The setting unit 42 reads endoscope information from the memory 25 when the endoscope 2 and the processor 4 are connected and the power of the processor 4 is turned on. Thereafter, the setting unit 42 sets the parallax image PL as the main parallax image based on the endoscope information read from the memory 25 (step S1 in FIG. 3), and sets the parallax image PR as a sub parallax image. . Then, the setting unit 42 outputs setting information indicating the main parallax image and the sub parallax image set as described above to the parallax image processing unit 43, the display control unit 44, and the dimming unit 45, respectively.

  The parallax image processing unit 43 specifies the parallax image PL as the main parallax image among the two parallax images output from the parallax image generation unit 41 based on the setting information output from the setting unit 42, and the parallax image PR is identified as a sub parallax image.

  The parallax image processing unit 43 acquires a white balance coefficient corresponding to the color tone of the parallax image PL, and performs white balance processing using the acquired white balance coefficient on the parallax image PL to the display control unit 44. Output (step S2 in FIG. 3). In addition, the parallax image processing unit 43 performs the white balance processing on the parallax image PR using the same white balance coefficient as that obtained when the white balance processing is performed on the parallax image PL, and the display control unit 44.

  Note that the white balance coefficient corresponding to the color tone of the parallax image PL is a coefficient for displaying the white area included in the parallax image PL in white on the display device 5, for example, the red color included in the parallax image PL. It is acquired as a combination of gain values individually multiplied with the luminance values of the component, green component and blue component.

  The display control unit 44 specifies the parallax image PL as the main parallax image out of the two parallax images output from the parallax image processing unit 43 based on the setting information output from the setting unit 42, and the parallax image PR. Are identified as sub parallax images.

  The display control unit 44 sequentially stores the parallax images PL and PR output simultaneously from the parallax image processing unit 43 in the memory 44A one by one. Further, the display control unit 44 calculates the blur amount in the blur evaluation area set for each of the N parallax images PL stored in the memory 44A during the period when the freeze switch of the input device 6 is turned off. Perform the process. Then, the display control unit 44 combines one parallax image PLF having the minimum blur amount among the N parallax images PL stored in the memory 44A and the memory 44A in a state combined with the one parallax image PLF. Are identified respectively.

  The display control unit 44 detects whether the current display mode is set to 3D display or 2D display based on an instruction from the display changeover switch of the input device 6 (step S3 in FIG. 3). When the display control unit 44 detects that the current display mode is set to 3D display (S3: YES), the display control unit 44 continues the operation in step S4 in FIG. When the display control unit 44 detects that the current display mode is set to 2D display (S3: NO), the display control unit 44 continues the operation of step S7 in FIG.

  When the current display mode is 3D display, the display control unit 44 performs an operation for causing the display device 5 to simultaneously display two parallax images of the parallax images PL and PR as observation images for stereoscopic viewing (see FIG. 3), it is detected whether or not the freeze switch of the input device 6 is turned on (step S5 in FIG. 3).

  When the display control unit 44 detects that the freeze switch of the input device 6 is turned on during 3D display (S5: YES), the display control unit 44 uses the parallax images PLF and PRF for stereoscopic vision instead of the parallax images PL and PR. An operation for simultaneously displaying the image as a freeze image on the display device 5 is performed (step S6 in FIG. 3). In addition, when the display control unit 44 detects that the freeze switch of the input device 6 is turned off during 3D display (S5: NO), the display control unit 44 continuously performs the operation for displaying the parallax images PL and PR simultaneously. .

  When the current display mode is 2D display, the display control unit 44 performs an operation for displaying the parallax image PL as an observation image on the display device 5 (step S7 in FIG. 3) and freezes the input device 6. It is detected whether or not the switch is turned on (step S8 in FIG. 3).

  When the display control unit 44 detects that the freeze switch of the input device 6 is turned on during 2D display (S8: YES), the display control unit 44 uses the parallax image PLF as the freeze image instead of the parallax image PL on the display device 5. An operation for displaying is performed (step S9 in FIG. 3). In addition, when the display control unit 44 detects that the freeze switch of the input device 6 is turned off during 2D display (S8: NO), the display control unit 44 continues the operation for displaying the parallax image PL.

  On the other hand, the dimming unit 45 specifies the parallax image PL as the main parallax image among the two parallax images output from the parallax image generation unit 41 based on the setting information output from the setting unit 42, and the parallax The image PR is specified as a sub parallax image.

  The light control unit 45 performs control for adjusting the light amount of the illumination light according to the brightness of the parallax image PL on the light source device 3 (step S10 in FIG. 3).

  Specifically, for example, the dimming unit 45 performs processing for calculating an average luminance value in the brightness evaluation region set in the parallax image PL, and calculates the calculated average luminance value and a predetermined target luminance value. Control for increasing, decreasing, or maintaining the amount of illumination light is performed on the light source device 3 in accordance with the difference value.

  That is, the operation of the dimming unit 45 in step S10 in FIG. 3 is an operation performed in parallel with the operations in steps S2 to S9 in FIG. 3 and is performed in both 3D display and 2D display. is there.

  Note that the above-described brightness evaluation region may be set, for example, in the entire area of the parallax image PL as long as an appropriate average luminance value corresponding to the brightness of the parallax image PL is calculated. The brightness evaluation area described above is preferably set to an area including the central portion of the parallax image PL.

  Here, according to the operation of the parallax image processing unit 43 as described above, the white balance coefficient acquired according to the color tone of the parallax image PL is displayed on the display device 5 during 3D display. It is reflected as the color tone of the parallax image PR included in the image (observation image). That is, according to the operation of the parallax image processing unit 43 as described above, the white balance coefficient applied to the two parallax images PL and PR is shared, so that the white for the two parallax images is white. The circuit scale or processing time required for the balance process can be reduced.

  Further, according to the operation of the display control unit 44 as described above, the amount of blur or color misregistration in the evaluation area set for each of the N parallax images PL stored in the storage unit 44 during 3D display. Is reflected as a selection factor of the parallax image PRF included in the display image (freeze image) displayed on the display device 5. That is, according to the operation of the display control unit 44 as described above, for example, it is troublesome to individually calculate blur amounts of N parallax images PL and N parallax images PR stored in the memory 44A. An image with less blur can be displayed on the display device 5 as a frozen image for stereoscopic viewing without performing any processing. Alternatively, according to the operation of the display control unit 44 as described above, for example, the color shift amounts of the N parallax images PL and the N parallax images PR stored in the memory 44A are individually calculated. Even without complicated processing, an image with little color shift can be displayed on the display device 5 as a freeze image for stereoscopic viewing.

  Further, according to the operation of the light control unit 45 as described above, the amount of illumination light supplied from the light source device 3 to the endoscope 2 changes, and the parallax image generated by the parallax image generation unit 41. The brightness of PL and PR changes simultaneously. Therefore, according to the operation of the light control unit 45 as described above, an average luminance value indicating the brightness in the evaluation region set in the parallax image PL is displayed on the display device 5 during 3D display. It is reflected as the brightness of the parallax image PR included in the work image (observation image). That is, according to the operation of the light control unit 45 as described above, for example, a complicated process of individually calculating the average luminance values of the parallax images PL and PR output from the parallax image generation unit 41 is performed. At least, the amount of illumination light supplied from the light source device 3 to the endoscope 2 can be adjusted.

  Therefore, according to the present embodiment, the operation as described above is performed in at least one of the parallax image processing unit 43, the display control unit 44, and the dimming unit 45, thereby the main and sub parallax images. Can be reduced in circuit scale or processing time required for 3D display.

  Note that the present embodiment is not limited to a configuration in which two parallax images corresponding to two optical images formed according to light from a subject are generated by the parallax image generation unit 41. For example, The present invention is also applied to the case where the parallax image generation unit 41 has a configuration in which three or more parallax images corresponding to three or more optical images formed according to light from the subject are generated. In such a configuration, the setting unit 42 sets one parallax image among the three or more parallax images generated by the parallax image generation unit 41 as the main parallax image, and the main parallax Two or more parallax images other than images may be set as sub parallax images.

  Further, according to the present embodiment, for example, the position of the protrusion 23B on the distal end surface 2B, the distance DL between the optical axis OL of the objective optical system 22L and the protrusion 23B, and the optical axis OR of the objective optical system 22R. The layout information of the distal end portion of the insertion portion 2 </ b> A including information that can specify the distance DR between the projection portion 23 </ b> B and the protrusion 23 </ b> B may be stored in the memory 25. Furthermore, according to the present embodiment, for example, the setting unit 42 determines whether two parallax images generated by the parallax image generation unit 41 are based only on the layout information or based on both the endoscope information and the layout information. One of the parallax images may be set as the main parallax image.

  On the other hand, by appropriately modifying the configuration according to the present embodiment, for example, the setting unit 42 converts the main and sub parallax images initially set based on the endoscope information read from the memory 25 into the endoscope. You may make it change according to other factors other than information.

  Specifically, for example, the setting unit 42 may passively change the main and sub parallax images in accordance with an instruction from the input device 6 operated by the user. Alternatively, for example, the setting unit 42 based on the two parallax images output from the parallax image generation unit 41, information indicating the projection amount from the projection port 23B of the distal end portion of the treatment instrument 8, and / or the projection port 23B. Information indicating the visibility of the treatment portion 81 of the treatment instrument 8 protruding from the image may be acquired, and the main and sub parallax images may be actively changed according to the acquired information.

  In a configuration in which the initial settings of the main and sub parallax images are changed according to factors other than the endoscope information, for example, the parallax image processing unit 43, the display control unit 44, and the light control In each unit of the unit 45, an operation corresponding to the changed main parallax image may be performed. The operation in such a case will be described using an example in which the main parallax image is changed from the parallax image PL to PR. In the following, a specific description regarding a portion to which the above-described operation or the like can be applied will be omitted as appropriate.

  When the parallax image processing unit 43 detects that the main parallax image has been changed from the parallax image PL to PR based on the setting information output from the setting unit 42, the parallax image processing unit 43 discards the existing white balance coefficient, A white balance coefficient corresponding to the color tone of the image PR is acquired, and a white balance process using the acquired white balance coefficient is performed on the parallax image PR and output to the display control unit 44. The parallax image processing unit 43 performs the white balance processing on the parallax image PL using the same white balance coefficient as that obtained when the white balance processing is performed on the parallax image PR, and the display control unit 44.

  Note that the white balance coefficient corresponding to the color tone of the parallax image PR is a coefficient for displaying the white region included in the parallax image PR in white on the display device 5, for example, the red color included in the parallax image PR. It is acquired as a combination of gain values individually multiplied with the luminance values of the component, green component and blue component.

  The display control unit 44 detects based on the setting information output from the setting unit 42 that the main parallax image has been changed from the parallax image PL to PR during the period when the freeze switch of the input device 6 is turned off. At this time, the calculation result of the blur amount existing is discarded, and a process for calculating the blur amount in the blur evaluation area set for each of the N parallax images PR stored in the memory 44A is performed. Then, the display control unit 44 combines one parallax image PRG having the smallest blur amount among the N parallax images PR stored in the memory 44A and the memory 44A in a state combined with the one parallax image PRG. One parallax image PLG stored in each is identified.

  The blur evaluation area described above may be a variable area that is reset by the display control unit 44 according to the changed main parallax image, or may be re-set according to the changed main parallax image. It may be a fixed area that is not set.

  On the other hand, when the display control unit 44 detects that the main parallax image has been changed from the parallax image PL to PR based on the setting information output from the setting unit 42, the display control unit 44 replaces the existing processing with the existing processing. The calculation result of the color misregistration amount is discarded, the color misregistration amount in the color misregistration evaluation area set for each of the N parallax images PR stored in the memory 44A is calculated, In this case, a process for identifying a single main parallax image PRG that has the smallest color misregistration amount may be performed.

  The color misregistration evaluation area may be a variable area that is reset by the display control unit 44 according to the changed main parallax image, or according to the changed main parallax image. It may be a fixed area that is not changed.

  When the display control unit 44 detects that the freeze switch of the input device 6 is turned on during 3D display, the display control unit 44 displays the parallax images PLG and PRG as a stereoscopic freeze image instead of the parallax images PL and PR. An operation for simultaneously displaying on the device 5 is performed.

  When the current display mode is 2D display, the display control unit 44 performs an operation for causing the display device 5 to display the parallax image PR as an observation image (instead of the parallax image PL).

  When the display control unit 44 detects that the freeze switch of the input device 6 is turned on during 2D display, the display control unit 44 causes the display device 5 to display the parallax image PRG as a freeze image instead of the parallax image PR. I do. Further, the display control unit 44 continuously performs an operation for displaying the parallax image PR when it is detected that the freeze switch of the input device 6 is turned off during 2D display.

  When the light control unit 45 detects that the main parallax image has been changed from the parallax image PL to PR based on the setting information output from the setting unit 42, the dimming unit 45 discards the calculation result of the existing average luminance value. In addition, a process for calculating the average luminance value in the brightness evaluation region set in the parallax image PR is performed, and the amount of illumination light is set according to a difference value between the calculated average luminance value and a predetermined target luminance value. Control for increasing, decreasing or maintaining the light source device 3 is performed.

  The brightness evaluation area described above may be a variable area that is reset by the dimming unit 45 according to the changed main parallax image, or according to the changed main parallax image. It may be a fixed area that is not reset.

  On the other hand, when the main parallax image is changed from the parallax image PL to PR, the operation for displaying the parallax image PR as an observation image at the time of 2D display and the parallax image PRG at the time of 2D display are displayed as a freeze image. Each operation other than the operation for the above may be maintained as the operation corresponding to the parallax image PL which is the main parallax image at the time of initial setting.

  Note that the present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 1 Stereoscopic endoscope system 2 Endoscope 2A Insertion part 3 Light source device 4 Processor 5 Display apparatus 6 Input device 21 Illumination optical system 22 Imaging part 25 Memory 41 Parallax image generation part 42 Setting part 43 Parallax image processing part 44 Display control 44A Memory 45 Light control unit

Japanese Unexamined Patent Publication No. 2006-180934

Claims (8)

A parallax image generation unit configured to generate a plurality of parallax images corresponding to a plurality of optical images formed according to light from a subject;
A display control unit configured to perform an operation for displaying a display image according to the plurality of parallax images;
An image processing apparatus comprising:
One parallax image of the plurality of parallax images is set as a main parallax image, and one or more parallax images other than the main parallax image among the plurality of parallax images are set as sub parallax images. Having a setting unit configured to
To reflect a parameter obtained from the main parallax image in the sub parallax image included in the display image when the main parallax image and the sub parallax image are simultaneously displayed as the display image. An image processing apparatus characterized in that the operation is performed. A light control unit configured to adjust the amount of illumination light for illuminating the subject;
When the main parallax image and the sub parallax image are simultaneously displayed as the display image, the dimming unit is configured to emit the illumination light according to the brightness in the evaluation region set in the main parallax image. The image processing apparatus according to claim 1, wherein the amount of light is adjusted. The setting unit changes the main parallax image according to an instruction from an input device operated by a user or information on the subject detected based on the plurality of parallax images,
The said light control part resets the said evaluation area | region according to the main parallax image after a change, when detecting that the said main parallax image was changed by the said setting part. 2. The image processing apparatus according to 2. A storage unit capable of storing the plurality of parallax images;
The display control unit includes one main parallax image having a minimum blur amount based on the blur amount in the evaluation area set for each of the plurality of main parallax images stored in the storage unit, and the one main parallax image. The image processing apparatus according to claim 1, wherein one sub parallax image stored in the storage unit in a state of being combined with the parallax image is simultaneously displayed as the display image. The setting unit changes the main parallax image according to an instruction from an input device operated by a user or information on the subject detected based on the plurality of parallax images,
The said display control part resets the said evaluation area | region according to the main parallax image after a change, when detecting that the said main parallax image was changed by the said setting part. 5. The image processing apparatus according to 4. A storage unit capable of storing the plurality of parallax images;
The display control unit includes one main parallax image having a minimum color misregistration amount based on the color misregistration amount in the evaluation region set for each of the plurality of main parallax images stored in the storage unit, and the 1 The image processing apparatus according to claim 1, wherein one sub parallax image stored in the storage unit in a state of being combined with one main parallax image is simultaneously displayed as the display image. . The setting unit changes the main parallax image according to an instruction from an input device operated by a user or information on the subject detected based on the plurality of parallax images,
The said display control part resets the said evaluation area | region according to the main parallax image after a change, when detecting that the said main parallax image was changed by the said setting part. 6. The image processing apparatus according to 6. A parallax image processing unit configured to perform image processing on the plurality of parallax images;
The parallax image processing unit obtains a white balance coefficient corresponding to a color tone of the main parallax image, and performs white balance processing using the white balance coefficient on the main parallax image and the sub parallax image. Giving,
The image processing apparatus according to claim 1, wherein the display control unit simultaneously displays the main parallax image and the sub parallax image subjected to the white balance process as the display image.

Images