JP2013043007A - Focal position controller, endoscope, and focal position control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a focal position controller, an endoscope, a focal position control method, etc. that enable the control of the focal positions by deciding the conditions of an object within the body viewed from the direction of its photographing by an imaging optical system.SOLUTION: The focal position controller includes an image acquirer to acquire a body interior image containing the image of an object in the body via an imaging optical system (equivalent to an A/D converter 310), a decision part to decide the conditions of the object within the body viewed from the direction of photographing it by the image optical system according to the acquired image of the object in the body (equivalent to a forward counter decision part or a static decision part), and a focal position controller to control the focal position of the imaging optical system by controlling the drive of the imaging optical system based on the decision result by the decision part.

Description

  The present invention relates to a focus position control device, an endoscope device, a focus position control method, and the like.

  In an imaging apparatus such as an endoscope, a pan-focus image is required in order not to hinder the diagnosis of the operator. For this reason, the endoscope achieves such performance by using an optical system having a relatively large F number to increase the depth of field. However, in recent years, an imaging device having a high pixel number of about several tens to millions of pixels has been used also in an endoscope system. When the optical system is constant, the depth of field is determined by the size of the permissible circle of confusion. However, in a high-pixel image sensor, the permissible circle of confusion decreases with the pixel pitch, so the depth of field of the imaging device is narrow. Become. For this reason, Patent Document 1 proposes an endoscope apparatus in which an imaging unit of an endoscope is provided with a focal position driving unit that drives the focal position of an objective optical system and performs autofocus (hereinafter referred to as AF) on a subject. Has been.

JP-A-8-106060

  In normal observation with an endoscope, a deep digestive tract is observed as shown in FIG. 8A, and therefore the distance to the corresponding subject varies greatly depending on the position on the acquired image. For this reason, even if AF is operated, it is only a part of the image that is in focus, and the other part is blurred. Therefore, there is a possibility that the user's observation may be hindered because the focus is not focused on a region (for example, a lesioned part) that the user wants to observe. That is, since the AF operation may not be effective, it is necessary to appropriately control the stop / start of the AF operation.

  In Patent Document 1, switching of stop / start of AF operation is controlled by a switch provided in an operation unit. For this reason, the operator needs to stop / start the AF operation in addition to the conventional endoscope operation, which increases the complexity of the work.

  According to some aspects of the present invention, a focus position control device, an endoscope device, and an endoscope device capable of controlling a focus position according to a situation by determining situation information of an in-vivo subject with respect to an imaging direction of an imaging optical system, and A focal position control method and the like can be provided.

  One aspect of the present invention provides an image acquisition unit that acquires an in-vivo image including an image of the in-vivo subject by imaging the in-vivo subject via an imaging optical system, and the image at the time of imaging based on the acquired in-vivo image. A determination unit that determines the status information of the in-vivo subject with respect to the imaging direction of the imaging optical system, and the focus position of the imaging optical system is controlled by controlling the driving of the imaging optical system based on the determination result in the determination unit. And a focal position control unit that controls the focal position control device.

  In one aspect of the present invention, the status information of the in-vivo subject with respect to the imaging direction is determined based on the in-vivo image. Since the focal position of the imaging optical system is controlled based on the determination result, the focal position can be controlled according to the relative situation between the imaging direction of the imaging optical system and the in-vivo subject, and an image that is easy for the user to view is provided. Etc. becomes possible.

  According to another aspect of the present invention, an image acquisition unit that acquires an in-vivo image including an image of the in-vivo subject by imaging an in-vivo subject via an imaging optical system, and an image acquisition based on the acquired in-vivo image. A determination unit that determines the status information of the in-vivo subject with respect to the imaging direction of the imaging optical system at the time, and the drive of the imaging optical system is controlled based on the determination result in the determination unit. The present invention relates to an endoscope apparatus including a focal position control unit that controls a focal position and a focal position driving unit that drives the imaging optical system based on the control of the focal position control unit.

  According to another aspect of the present invention, an in-vivo image including an image of the in-vivo subject is acquired by imaging an in-vivo subject via an imaging optical system, and the imaging at the time of imaging is performed based on the acquired in-vivo image. Related to a focal position control method for controlling the focal position of the imaging optical system by determining status information of the in-vivo subject with respect to the imaging direction of the optical system and controlling the driving of the imaging optical system based on the determination result To do.

The structural example of the endoscope apparatus containing the focus position control apparatus concerning this embodiment. A configuration example of a rotating color filter. An example of spectral characteristics of a rotating color filter. 2 is a configuration example of an image processing unit. 2 is a configuration example of a focus lens control unit. Setting example of evaluation area. 4 is a configuration example of a contrast value calculation unit. FIG. 8A to FIG. 8F are examples of the positional relationship between the imaging optical system and the in-vivo subject and in-vivo images acquired in the positional relationship. The structural example of a confrontation determination part. The example of the gain map used for the image process with respect to an in-vivo image. FIG. 11A to FIG. 11C show examples of in-vivo images after processing using a gain map. 12A to 12C show examples of in-vivo images after low-frequency extraction. An example of region division of an in-vivo image in a dividing unit. The figure explaining the correspondence of the divided | segmented area | region and the luminance value calculated in the said area | region. FIGS. 15A to 15C are diagrams for explaining the relationship between the difference in brightness distribution according to the positional relationship between the imaging optical system and the in-vivo subject and the divided regions. The other structural example of the endoscope apparatus containing the focus position control apparatus concerning this embodiment. Another example of the configuration of the focus lens control unit. Another example of the configuration of the focus lens control unit. The structural example of a stillness determination part. Another example of the configuration of the focus lens control unit. An example of setting representative points on an in-vivo image. 22A to 22D are diagrams illustrating a vanishing point detection method.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

  The present applicant proposes a technique capable of executing AF in an appropriate state by determining situation information of the in-vivo subject with respect to the imaging direction of the imaging optical system, and specifically, in the first embodiment. Whether the in-vivo subject is directly facing the imaging optical system is used as the situation information. In the second embodiment, in addition to whether or not the in-vivo subject faces the imaging optical system, whether or not the imaging optical system (specifically, a zoom lens included in the imaging optical system) is at the telephoto end. The AF is controlled according to the above. In the third embodiment, the AF is controlled depending on whether the in-vivo subject is stationary relative to the imaging optical system and whether the imaging optical system is at the telephoto end. In the fourth embodiment, whether the in-vivo subject is directly facing the imaging optical system, whether the in-vivo subject is relatively stationary with respect to the imaging optical system, and is the imaging optical system at the telephoto end? The AF is controlled according to whether or not.

1. First Embodiment FIG. 1 shows a configuration example of an endoscope apparatus including a focal position control apparatus according to the present embodiment. The endoscope apparatus includes a light source unit 100, an imaging unit 200, a control device 300 (processor unit), a display unit 400, and an external I / F unit 500. The focal position control device according to the present embodiment corresponds to the control device 300, for example.

  The light source unit 100 includes a white light source 110 and a rotating color filter 120 having a plurality of spectral transmittance filters. The light source unit 100 also includes a rotation driving unit 130 that drives the rotary color filter 120 and a condensing lens 140 that condenses the light transmitted through the rotary color filter 120 on the incident end face of the light guide fiber 210. FIG. 2 shows a detailed configuration example of the rotating color filter 120. The rotary color filter 120 includes three primary colors of red (hereinafter abbreviated as R) filter 701, green (hereinafter abbreviated as G) filter 702, blue (hereinafter abbreviated as B) filter 703, and a rotary motor 704. Yes. FIG. 3 shows an example of spectral characteristics of these color filters 701 to 703. The rotation driving unit 130 rotates the rotation color filter 120 at a predetermined number of rotations in synchronization with the imaging period of the imaging element 260 based on a control signal from the control unit 340. For example, if the rotating color filter 120 is rotated 20 times per second, each color filter crosses incident white light at 1/60 second intervals. In this case, the image sensor 260 completes imaging and transfer of image signals at 1/60 second intervals. Here, the image sensor 260 is, for example, a monochrome image sensor. That is, in the present embodiment, frame sequential imaging is performed in which images of light of each of the three primary colors (R, G, or B) are captured at 1/60 second intervals.

  The imaging unit 200 is formed to be elongated and bendable, for example, to enable insertion into a body cavity. The imaging unit 200 diffuses the light guide fiber 210 for guiding the light collected by the light source unit 100 to the illumination lens 220 and the light guided to the tip by the light guide fiber 210 to observe an object (subject, Specifically, it includes an illumination lens 220 that irradiates an in-vivo subject). In addition, the imaging unit 200 includes an objective lens 230 that collects reflected light returning from the observation target, a focus adjustment lens 240 for adjusting the focus position, and a focus position drive unit 250 for driving the focus adjustment lens 240. The imaging device 260 for detecting the condensed reflected light is provided. The focal position driving unit 250 is a stepping motor, for example, and is connected to the focal adjustment lens 240. The focus position driving unit 250 adjusts the focus position by changing the position of the focus adjustment lens 240. The imaging element 260 is, for example, a monochrome single-plate imaging element, and is configured by, for example, a CCD or a CMOS image sensor.

  The control device 300 controls each part of the endoscope device and performs image processing. The control device 300 includes an A / D conversion unit 310 (an image acquisition unit in a broad sense), a focus lens control unit 320, an image processing unit 330, and a control unit 340.

  The image signal converted into a digital signal by the A / D conversion unit 310 is transferred to the image processing unit 330. The image signal processed by the image processing unit 330 is transferred to the focus lens control unit 320 and the display unit 400. The focus lens control unit 320 changes the position of the focus adjustment lens 240 by transferring a control signal to the focus position driving unit 250. The control unit 340 controls each unit of the endoscope apparatus. Specifically, the control unit 340 synchronizes the rotation driving unit 130, the focus lens control unit 320, and the image processing unit 330. The focus lens control unit 320 and the image processing unit 330 are controlled based on an input from the external I / F unit 500, which is connected to the external I / F unit 500.

  The display unit 400 is a display device that can display a moving image, and includes, for example, a CRT or a liquid crystal monitor.

  The external I / F unit 500 is an interface for performing input from an operator to the endoscope apparatus. The external I / F unit 500 includes, for example, a power switch for turning on / off the power, a mode switching button for switching a photographing mode and other various modes. The external I / F unit 500 transfers the input information to the control unit 340.

  FIG. 4 shows a detailed configuration example of the image processing unit 330 in the present embodiment. The image processing unit 330 includes a preprocessing unit 331, a synchronization unit 332, and a postprocessing unit 333.

  The A / D conversion unit 310 is connected to the preprocessing unit 331. The preprocessing unit 331 is connected to the synchronization unit 332. The synchronization unit 332 is connected to the post-processing unit 333 and the focus lens control unit 320. The post-processing unit 333 is connected to the display unit 400. The control unit 340 is connected to the pre-processing unit 331, the synchronization unit 332, and the post-processing unit 333, and performs these controls.

  The preprocessing unit 331 uses the OB clamp value, gain correction value, and WB coefficient value stored in advance in the control unit 340 for the image signal input from the A / D conversion unit 310, Gain correction processing and WB correction processing are performed. The preprocessing unit 331 transfers the preprocessed image signal to the synchronization unit 332.

  The synchronizer 332 synchronizes the frame sequential image signal with the image signal processed by the preprocessor 331 based on the control signal of the controller 340. Specifically, the synchronizer 332 accumulates image signals of each color light (R, G, or B) input in frame order one frame at a time, and simultaneously reads the accumulated image signals of each color light. The synchronization unit 332 transfers the synchronized image signal to the post-processing unit 333 and the focus lens control unit 320.

  The post-processing unit 333 uses the tone conversion coefficient, color conversion coefficient, and edge enhancement coefficient stored in advance in the control unit 340 for the image signal after synchronization, and performs tone conversion processing, color processing, and contour processing. Perform enhancement processing. The post-processing unit 333 transfers the post-processed image signal to the display unit 400.

  FIG. 5 shows a detailed configuration example of the focus lens control unit 320 in the present embodiment. The focus lens control unit 320 includes a confrontation determination unit 321, an evaluation region setting unit 322, a contrast value calculation unit 323, a focus position detection unit 324, and a focus position control unit 325.

  The image processing unit 330 is connected to the confrontation determination unit 321 and the evaluation area setting unit 322. The confrontation determination unit 321 is connected to the focal position control unit 325. The evaluation area setting unit 322 is connected to the contrast value calculation unit 323. The contrast value calculation unit 323 is connected to the focus position detection unit 324. The focus position detection unit 324 is connected to the focus position control unit 325. The focal position controller 325 is connected to the in-focus position detector 324 and the focal position driver 250. The control unit 340 is connected to the confrontation determination unit 321, the evaluation region setting unit 322, the contrast value calculation unit 323, the focus position detection unit 324, and the focus position control unit 325, and controls these I do.

  The facing determination unit 321 determines whether the subject is facing the imaging unit 200 based on the image signal transferred from the image processing unit 330. When the facing determination unit 321 determines that the subject is facing the imaging unit 200, the facing determination unit 321 transfers a control signal for starting the AF operation to the focal position control unit 325. On the other hand, when it is determined that the subject does not face the imaging unit 200, a control signal for stopping the AF operation is transferred to the focal position control unit 325. The process performed by the facing determination unit 321 will be described later in detail.

  The evaluation area setting unit 322 sets an evaluation area for calculating a contrast value for the image signal transferred from the image processing unit 330. Here, as shown in FIG. 6, an area set in advance may be set as an evaluation area, or an operator inputs information about the evaluation area from the external I / F unit 500, and an image is arbitrarily selected based on this information. The evaluation area may be set. Thereafter, the evaluation area setting unit 322 transfers the set evaluation area information and the image signal to the contrast value calculation unit 323.

  The contrast value calculation unit 323 calculates the contrast value of the evaluation area from the evaluation area information and the image signal. In this embodiment, the image signal transferred from the evaluation area setting unit 322 may calculate a contrast value for an arbitrary channel. Further, a luminance signal may be generated from pixel values of three channels of R, G, and B, and a contrast value may be calculated for the pixel value of the generated luminance signal. Here, for example, the contrast value calculation unit 323 may perform arbitrary high-pass filter processing on all the pixels included in the evaluation region, and calculate the contrast value by adding all the high-pass filter output values of each pixel. Further, the contrast value calculation unit 323 may be configured to include a bright spot removal unit 3231 before the high-frequency extraction unit 3232 that performs high-pass filter processing, for example, as illustrated in FIG. 7. Here, for example, the bright spot removal unit 3231 performs threshold processing on an arbitrary channel or pixel value of a luminance signal in all pixels included in the evaluation region, and determines that the pixel is a bright spot if the pixel value is equal to or greater than the threshold. By transferring a control signal for setting the output value of the subsequent high-pass filter processing to 0, the influence of the bright spot on the contrast value can be reduced. Thereafter, the contrast value calculation unit 323 transfers the contrast value of the evaluation region to the focus position detection unit 324.

  When the focus position detection unit 324 receives the control signal for detecting the focus position from the focus position control unit 325, the focus position detection unit 324 detects the focus position based on the contrast value transferred from the contrast value calculation unit 323. Here, the in-focus position is position information of the focus adjustment lens 240 that is considered to be in focus on the subject. Here, in order to detect the in-focus position, the contrast value peak is detected using the contrast information sequentially transferred from the contrast value calculation unit 323. The focus position detection unit 324 transfers the peak position of the contrast value to the focus position control unit 325 as the focus position.

  The focus position control unit 325 transfers the position information to the focus position driving unit 250 and changes the position of the focus adjustment lens 240. Specifically, when the control signal for starting the AF operation is transferred from the facing determination unit 321, the focus position control unit 325 transfers a control signal for detecting the focus position to the focus position detection unit 324. Further, the focus position driving unit 250 is controlled so that the position of the focus adjustment lens 240 is gradually changed using an AF method such as a known hill-climbing method. When the focus position information is transferred from the focus position detection unit 324, the focus position control unit 325 moves the focus adjustment lens 240 to the focus position, and ends the AF operation. On the other hand, the focus position control unit 325 moves the focus adjustment lens 240 to a predetermined position stored in advance in the control unit 340 when a control signal for stopping the AF operation is transferred from the facing determination unit 321. The focal position driving unit 250 is controlled.

  The relative positional relationship between the subject and the image pickup unit and the relationship between the image signals will be described in detail with reference to FIGS.

  As shown in FIG. 8A, when a lesion is screened in a tubular subject (organ), observation is performed while the scope is parallel to the subject and the scope is inserted or removed. However, when the AF operation is performed on the subject having such depth, the focus position does not always coincide with the region in which the operator is interested, so it is desirable not to perform the AF operation. As shown in FIG. 8B, the image signal at this time is darker at the center of the image and becomes brighter toward the periphery of the image. On the other hand, as shown in FIG. 8C, when there is a region in which the surgeon is interested in the subject, the surgeon tilts the scope in a direction facing the subject in order to bring the scope closer to the subject. Even in such a case, since the subject has a depth with respect to the imaging unit, it is desirable not to perform the AF operation. As shown in FIG. 8D, the image signal at this time is an image having a contrast difference on the diagonal of the image. Also, as shown in FIG. 8E, when the surgeon examines the subject, the surgeon faces the scope to the subject. In such a case, it is desirable to perform the AF operation because the depth of the subject is sufficiently small with respect to the imaging unit. As shown in FIG. 8F, the image signal at this time becomes an image that is brighter at the center of the image and darker toward the periphery of the image.

  FIG. 9 shows a detailed configuration example of the confrontation determination unit 321 in the present embodiment. The confrontation determination unit 321 includes a correction unit 3211, a low frequency extraction unit 3212, a division unit 3213, a brightness calculation unit 3214, a comparison unit 3215, and a determination unit 3216.

  The correction unit 3211 corrects the shading of the optical system for the image signal transferred from the image processing unit 330. In general, an optical system of an endoscope has a shading characteristic that becomes darker as the distance from the center of an image increases. Here, shading correction is performed by multiplying the image signal by a gain map stored in advance in the control unit 340. A gain map used for shading correction is shown in FIG. Further, FIG. 11 (A) to FIG. 11 (C) show image signals obtained by performing shading correction on the image signals shown in FIG. When the imaging unit 200 is positioned in parallel with a tubular subject, as shown in FIG. 11A, the contrast of the image signal is larger than before the correction. Further, when the subject is located obliquely with respect to the imaging unit 200, as shown in FIG. 11B, the brightness difference of the image signal is not eliminated. On the other hand, when the subject faces the imaging unit 200, the brightness of the image signal is uniform as shown in FIG. The image signal after the shading correction is transferred to the low-frequency extraction unit 3212.

  The low frequency extraction unit 3212 extracts a low frequency component from the image signal transferred from the correction unit 3211. Here, a low frequency component of the image signal is extracted by applying a low pass filter to the image signal. Specifically, a Gaussian filter is applied to the image signal. Image signals obtained by extracting low-frequency components from the image signals shown in FIG. 11 are shown in FIGS. The image signal from which the low-frequency component has been extracted becomes an image showing the brightness distribution of the subject from which fine structures (such as blood vessels) included in the image signal are removed. Here, the Gaussian filter is applied as the process of extracting the low-frequency component. However, the present invention is not limited to this. For example, a known smoothing filter such as an averaging filter or a median filter may be used. Alternatively, the high-frequency component may be removed by reducing the image signal, and the low-frequency component may be extracted by expanding the reduced image signal to the original size. The image signal from which the low frequency component has been extracted is transferred to the dividing unit 3213.

  The dividing unit 3213 divides the image signal transferred from the low-frequency extracting unit 3212 into regions. Specifically, as shown in FIG. 13, the image signal is divided into nine parts, an image center region and a region located on the outer periphery thereof. Here, the area to be divided is nine divisions, but is not limited thereto, and may be five divisions or 17 divisions. The region-divided image signal is transferred to the brightness calculation unit 3214.

  The brightness calculation unit 3214 calculates the brightness for each area of the image signal transferred from the dividing unit 3213. Specifically, a value obtained by averaging the luminance values Y (x, y) of the pixels (x, y) included in each area is calculated as the brightness Y of the area. Here, as shown in FIG. 14, the brightness of the region i is expressed as Yi. The calculated brightness Yi of each region is transferred to the comparison unit 3215.

  The comparison unit 3215 compares the brightness Yi transferred from the brightness calculation unit 3214 to calculate the maximum value of the contrast between regions. Specifically, first, the contrast D1 between the center of the image and the periphery is calculated as shown in the following equation (1).

  Subsequently, as shown in the following equation (2), contrast differences D2 to D5 in the diagonal direction around the image are calculated.

  In the above equation (2), abs (A) is a function that returns the absolute value of A. Thereafter, the maximum value Dmax of the light / dark differences D1 to D5 between the regions is calculated. Here, when the imaging unit 200 is positioned in parallel to the luminal subject, as shown in FIG. 15A, the contrast between the image center and the periphery becomes the maximum value Dmax. Further, when the subject is located obliquely with respect to the imaging unit 200, as shown in FIG. 15B, the contrast difference in the diagonal direction becomes the maximum value Dmax. On the other hand, when the subject is directly facing the imaging unit 200, as shown in FIG. 15C, one of the light and dark differences D1 to D5 has the maximum value Dmax, but the maximum value Dmax is the value of FIG. A) and a sufficiently small value as compared with the case of FIG. The maximum value Dmax of the light / dark difference is transferred to the determination unit 3216.

  The determination unit 3216 determines whether or not the subject is facing the imaging unit based on the maximum value Dmax of the contrast difference transferred from the comparison unit 3215. Specifically, when the maximum value Dmax of the contrast difference is smaller than the threshold value ThD stored in advance in the control unit 340, it is determined that the subject is facing the imaging unit 200. The determination unit 3216 transfers a control signal for starting the AF operation to the focus position control unit 325 when the subject changes from the state where the subject is not facing the imaging unit 200 to the state where the subject is facing the imaging unit 200. On the other hand, the determination unit 3216 transfers a control signal for stopping the AF operation to the focus position control unit 325 when the subject changes from the state facing the imaging unit to the state not facing the imaging unit. Here, in order to stably perform the AF operation, the control signal may be transferred when the state of the subject is changed and the state is maintained for several frames.

  As described above, in the present embodiment, it is determined whether or not the imaging unit is facing the subject from the image signal. If it is determined that the imaging unit is facing the subject, the AF operation is started. On the other hand, when it is determined that the imaging unit is not facing the subject, the AF operation is stopped. Thereby, the surgeon does not need to perform the start / stop operation of the AF operation, and can obtain a suitable image according to the scene.

  In the above description, the single AF is assumed as the AF operation. However, the present invention is not limited to this. For example, the continuous AF may be used as the AF operation. In this case, the overall configuration is the same as that described above, but the operation of the focus lens control unit 320 is different.

  When the control signal for starting the AF operation is transferred from the facing determination unit 321, the focal position control unit 325 starts the continuous AF. On the other hand, the focus position control unit 325 moves the focus adjustment lens 240 to a predetermined position stored in advance in the control unit 340 when a control signal for stopping the AF operation is transferred from the facing determination unit 321. The focal position driving unit 250 is controlled.

Below, the flow of the continuous AF of this embodiment is described. Here, the reciprocal width during wobbling is assumed to be ± dw, and the movement width of the focus adjustment lens 240 (update value of the focal position) is assumed to be dn. First, the focus position control unit 325 changes the focus position of the focus adjustment lens 240 to ds−dw via the focus position drive unit 250 (ds is the initial position of the focus adjustment lens 240). Further, the focus position control unit 325 transfers the position information ds-dw of the focus adjustment lens 240 to the focus position detection unit 324. Then, the contrast value calculation unit 323 calculates the contrast value C- _dw . The calculated contrast value C _-dw is transferred to the focus position detection unit 324. Next, the focal position control unit 325 changes the focal position of the focus adjustment lens 240 to ds + dw via the focal position driving unit 250. Further, the focus position control unit 325 transfers the position information ds + dw of the focus adjustment lens 240 to the focus position detection unit 324. Then, the contrast value calculation unit 323 calculates the contrast value C _{+ dw} . The calculated contrast value C _{+ dw} is transferred to the in-focus position detection unit 324.

After calculating C _−dw and C _{+ dw} , the focus position detection unit 324 determines the initial focus position based on the position information transferred from the focus position control unit 325 and the contrast value transferred from the contrast value calculation unit 323. The value ds is updated according to the following conditions. If C _-dw > C _{+ dw} , the value of ds is decreased by dn, and if C _{+ dw} > C _-dw , the value of ds is increased by dn.

  The reciprocating width dw at the time of wobbling is subtracted from the updated initial position ds of the focus adjustment lens 240, and the focus position ds−dw is transferred to the focus position control unit 325. Thereafter, similar processing is repeated.

  Note that the above-described values of dw and dn may be set in advance, or the user may set arbitrary values from the external I / F unit 500.

  In addition, when performing continuous AF, the operation of the confrontation determination unit 321 is also different, and a control signal for performing the AF operation is always sent during the confrontation for continuous AF. When the determination unit 3216 determines that the subject is directly facing the imaging unit 200, the determination unit 3216 transfers a control signal for starting the AF operation to the focus position control unit 325. On the other hand, when the determination unit 3216 determines that the subject is not directly facing the imaging unit, the determination unit 3216 transfers a control signal for stopping the AF operation to the focus position control unit 325.

  The imaging optical system controlled by the focal position control apparatus according to the present embodiment is a lens group driving type that changes the angle of view (imaging magnification) and adjusts the focus at the same time by operating the zoom lens. Assumed. However, the present invention is not limited to this, and an imaging optical system driven by two lens groups that can independently adjust the positions of the zoom lens and the focus lens may be used.

  In the above-described embodiment, the focal position control apparatus acquires an in-vivo image including an in-vivo subject image via the imaging optical system (including the objective lens 230 and the focus adjustment lens 240 in FIG. 1) ( 1) and a determination unit that determines the status information of the in-vivo subject with respect to the imaging direction of the imaging optical system at the time of imaging based on the in-vivo image (a directivity determination unit 321 in FIG. 5). 18) and a focal position control unit 325 (FIG. 5) that controls the focal position of the imaging optical system based on the determination result in the determination unit.

  Here, the imaging direction of the imaging optical system is a direction in which imaging is performed using the imaging optical system, and may be the optical axis direction of the imaging optical system. Further, the status information of the in-vivo subject with respect to the imaging direction is information indicating the status of the in-vivo subject with respect to the imaging direction. Therefore, the state information of the in-vivo subject that does not depend on the imaging direction (for example, whether or not a lesion is included or whether or not the blood vessel structure is concentrated) is not included in the situation information in this embodiment. However, information such as the lesion is not necessarily included in the situation information. For example, the size and shape of the lesion changes according to the imaging direction, and the determination necessary for focal position control can be performed based on the change. If so, information on the lesion may be included in the situation information.

  Thereby, it becomes possible to control the focal position of the imaging optical system based on the status information of the in-vivo subject with respect to the imaging direction of the imaging optical system. Specifically, the control of the focal position may be an operation for focusing on the in-vivo subject (AF operation) as described later. In this case, the AF operation can be controlled according to the situation information, and the AF can be operated in an appropriate situation. As described above, in the situation shown in FIG. 8A, the distance from the imaging optical system to the subject varies greatly depending on the position of the in-vivo image (captured image) on the image. In that case, even if the AF is operated, only a part of the in-vivo image is focused, and there is a possibility that an area that the user wants to observe is blurred. Therefore, the AF is not always operated, but may be operated when it is determined that the AF is an effective situation based on the situation information.

  The determination unit may determine at least one of posture information of the in-vivo subject with respect to the imaging direction of the imaging optical system and movement information of the in-vivo subject with respect to the imaging direction of the imaging optical system as the status information of the in-vivo subject.

  Here, the posture information may be information representing, for example, an angle formed by an in-vivo subject (a surface of a living body to be imaged) with respect to an imaging direction. For example, FIG. 8A illustrates a situation where the in-vivo subject is parallel to the imaging direction, and FIG. 8B illustrates an in-vivo subject that is oblique to the imaging direction. In FIG. 8C, the in-vivo subject is in a vertical or close posture with respect to the imaging direction (this state is referred to as “facing” as will be described later). Information representing such a difference in posture is posture information. The movement information is information that represents the relative movement between the imaging optical system and the in-vivo subject in association with the imaging direction. Specifically, information indicating the direction of movement and the amount of movement can be considered. In that case, the direction of motion is expressed as the direction relative to the reference direction with the imaging direction as the reference direction.

  This makes it possible to specifically use at least one of posture information and motion information as the status information of the in-vivo subject. That is, in the present embodiment, the relative positional relationship information between the imaging optical system and the in-vivo subject (situation information itself or information indicating a change in positional relationship) may be used as the situation information. Is used. As described above with reference to FIG. 8A and the like, it is assumed that whether or not the focus position control (especially the AF operation) is effective is determined according to the positional relationship information. By using the positional relationship information, it is possible to control the focal position appropriately according to the situation.

  The determination unit may include a confrontation determination unit 321 that performs determination using information regarding whether or not the in-vivo subject is directly facing the imaging optical system as the above-described posture information. Then, the focal position control unit 325 controls the focal position of the imaging optical system based on the determination result in the facing determination unit 321.

  Here, that the in-vivo subject is directly facing the imaging optical system means that the imaging direction (optical axis direction) of the imaging optical system is orthogonal to the in-vivo subject (for example, the plane if the surface of the living body can be regarded as a plane). It represents the state to do. However, it is not necessarily limited to the orthogonal state, and when the given angle threshold value is x (°), the angle formed by the imaging direction and the in-vivo subject is within the range of 90 ± x (°). If so, it may be determined that they are facing each other. The angle threshold value x may be determined by the system, or may be input by the user from the external I / F unit 500 or the like. In this embodiment, as described above, it is determined whether or not it is facing directly using the brightness information for each divided area, and the threshold value used for the determination is a value represented by the brightness information. It is related to the difference value. That is, in the focal position control apparatus of the present embodiment, a case where the above-described angle threshold value x is not used can be considered sufficiently. In this case as well, whether the imaging direction and the in-vivo subject are correct or not are determined. It is still determined that the determination is based on whether or not the angle is orthogonal or close to orthogonal.

  Accordingly, it becomes possible to control the focal position of the imaging optical system based on whether or not it is determined that the in-vivo subject is facing the imaging optical system. In the example of FIG. 8A and the like, it is determined that the states of FIG. 8A and FIG. 8C are not facing each other, and the state of FIG. . In view of the fact that the region corresponding to the inside of the two dotted lines in each figure is captured as an in-vivo image, it corresponds to the pixel position on the in-vivo image compared to the case where it is not facing, The difference in distance to the subject is reduced. That is, it is assumed that when the camera is facing the camera, a wide area of the in-vivo image is focused, whereas when the camera is not facing the camera, only the narrow area of the in-vivo image is focused. Therefore, it is possible to provide an image that is easy for the user to view by appropriately controlling the focal position according to such a difference in situation.

  In addition, the focus position control unit 325 starts an operation for focusing on the in-vivo subject (focusing operation) when it is determined by the confrontation determination unit 321 that the in-vivo subject is facing the imaging optical system. May be. When it is determined that the in-vivo subject is not directly facing the imaging optical system, the focusing operation is stopped and an imaging optical system (specifically, a zoom lens included in the imaging optical system) is given. It may be moved to the position.

  Here, the given position to which the imaging optical system is moved when not facing each other may be determined by the system, or may be input by the user from the external I / F unit 500 or the like. The case where it is not facing is a case as shown in FIG. 8A, and it is assumed that the situation is such as screening using an endoscope apparatus. In this case, as described above, an image is taken from a near subject to a far subject, and the subject distance difference according to the position on the image is large, so that pan focus is necessary. Further, in the case of screening, the zoom magnification is not so necessary. That is, as an example of the process of moving the imaging optical system to a given position, it is a movement to a position that realizes pan focus and low magnification. If the lens group is driven, a zoom lens (the focus adjustment lens in FIG. This corresponds to a process of moving the position corresponding to 240 to the telephoto side (for example, the telephoto end) to some extent. The movement to a given position is not limited to the zoom lens, and the zoom lens position may be moved, and another lens included in the imaging optical system may be moved. In the case of lens two-group drive, the focus position can be controlled by controlling the focus lens position independently of the zoom lens. In the case of the above-described screening or the like, it is assumed that a certain wide range is imaged in order to find a lesioned part or the like, and the distance between the imaging optical system and the in-vivo subject is larger than that during magnified observation or the like. Therefore, even if the depth of field is increased by moving the zoom lens position, if the depth of field is set to a position close to the imaging optical system, the subject may not be in focus and observation may be difficult There is sex. In order to deal with such a case, in the second lens group driving, the zoom lens position is moved, the focus lens position is also moved, and the depth of field is set to a position somewhat away from the imaging optical system.

  This makes it possible to control the focusing operation on the in-vivo subject being imaged (that is, corresponding to AF control) depending on whether or not the camera is directly facing. As described above, in the situation shown in FIG. 8A, even if AF is performed, only a partial area on the image is in focus, so it is difficult to determine the area that the user wants to observe from the image. Therefore, even if AF is executed, it is not effective. In other words, by controlling the AF operation based on the determination of whether or not the camera is facing, it is possible to operate the AF when the AF is effective, and not to operate the AF when the AF is not effective. The user's observation (such as diagnosis using an endoscope apparatus) can be made smooth.

  The facing determination unit 321 may include a brightness calculation unit 3214 that calculates brightness information based on the image signal of the in-vivo image. Then, the facing determination unit 321 determines whether or not the in-vivo subject is facing the imaging optical system based on the brightness information.

  Here, the brightness information is a value obtained based on the luminance value of the pixel included in the image signal, but is not limited thereto. If three channels of R, G, and B are used, it may be obtained based on the pixel value of any one channel, or may be obtained by another method.

  As a result, it is possible to determine whether or not they are facing each other based on the brightness information of the image signal. As described above, the state of facing directly refers to the case where the imaging direction and the in-vivo subject are orthogonal or close to each other (within a given angle range). It is difficult to actually ask. By using the brightness information, it is possible to estimate the positional relationship between the imaging direction (optical axis) and the in-vivo subject without obtaining a strict angle value, thereby facilitating processing.

  The facing determination unit 321 may include a dividing unit 3213 that divides the in-vivo image into a plurality of regions. The brightness calculation unit 3214 calculates brightness information for each divided area. The facing determination unit 321 determines whether the in-vivo subject is facing the imaging optical system based on the brightness information calculated for each region.

  Here, the division of the in-vivo image in the dividing unit 3213 may be, for example, divided into nine regions as shown in FIG. Moreover, it is not limited to 9 areas, You may perform different division | segmentation.

  Thereby, it is possible to obtain brightness information after dividing the in-vivo image into a plurality of regions and use the brightness information for the facing determination. As shown in FIG. 12A to FIG. 12C, the brightness distribution tendency in the in-vivo image changes depending on whether or not it is directly facing. FIGS. 12A to 12C are those after the processing by the gain map of FIG. 10 and the low-frequency extraction processing are performed. When facing directly, the brightness is almost uniform as shown in FIG. 12C, whereas when not facing, it is bright at the center and the periphery as shown in FIG. The brightness is greatly different between the upper part and the lower part as shown in FIG. 12B (the difference in brightness in the left and right or diagonal directions may be used for FIG. 12B). . That is, whether or not it is facing is determined by looking at the difference value of brightness information between a certain area on the in-vivo image and another area. As an actual technique, the area division by the dividing unit 3213 is performed. Will do.

  The dividing unit 3213 may divide the in-vivo image into a central region and at least one peripheral region. Then, based on the comparison processing of the brightness information of the center area and the brightness information of the peripheral area, the in-body subject and the imaging optical system are determined to face each other.

  Here, for example, the central region corresponds to the region 1 in FIG. 13, and the peripheral region corresponds to the regions 2 to 9 in FIG. However, it is not limited to the case where the entire region of the in-vivo image is included in either the central region or the peripheral region, and there may be a region that is not included in the central region or the peripheral region.

  As a result, it is possible to determine whether the person is facing the center area and the peripheral area. Specifically, it becomes possible to determine that the situation as shown in FIG. When the imaging direction is parallel to the tube wall as shown in FIG. 8A, the central region of the in-vivo image becomes dark because it corresponds to a subject far from the imaging optical system, and the peripheral region has a distance. The result is a comparison of the brightness information of the center area and the peripheral area (specifically, whether the difference between the values represented by the brightness information is greater than a given threshold) Based on (comparison result), it becomes distinguishable from the case where it is facing (FIG. 12C). Specifically, this corresponds to using D1 of the formula (1) for the determination.

  The dividing unit 3213 may divide the in-vivo image into a central region and first to Nth (N is an integer of 2 or more) peripheral regions. Then, the right-facing determination unit 321 includes the i-th (i is an integer of 1 ≦ i <N) to the j-th (j is an integer of i ≦ j <N) of the first to Nth peripheral regions. And brightness in the kth (m is an integer of j <k ≦ N) to mth (m is an integer of k ≦ m ≦ N) out of the first to Nth peripheral regions. Based on the information comparison process, the in-body subject and the imaging optical system are determined to face each other.

  Here, for example, the central region corresponds to the region 1 in FIG. 13 and the first to Nth peripheral regions correspond to the regions 2 to 9 in FIG. 13 (that is, N = 8 and the zth peripheral region). Is the region z + 1 in FIG. 13). Since 1 ≦ i ≦ j <k ≦ m ≦ N, for example, when i = 1, j = 4, k = 5, m = 8, the brightness information of the regions 2 to 5 and the regions 6 to 9 brightness information is compared, and the difference in brightness between the right side and the left side is determined. Note that the starting points of the first to Nth peripheral regions (the positions of the first peripheral regions) are not limited to those in FIG. 13, and may be started from arbitrary positions. For example, it may be rotated 90 ° clockwise with reference to FIG. 13. In this case, the regions 2 to 5 correspond to the lower half, the regions 6 to 9 correspond to the upper half, and The difference in brightness is determined. Similarly, by setting the clockwise rotation to 45 ° or 135 °, it becomes possible to compare diagonal directions such as lower right and upper left, lower left and upper right. Moreover, what is necessary is just to satisfy | fill the above-mentioned conditions about i, j, k, and m. Therefore, the number of peripheral regions included in the i-th to j-th peripheral regions does not need to match the number of peripheral regions included in the k-th to m-th peripheral regions. There may be a peripheral region that is not included in the i-th to j-th and k-th to m-th peripheral regions.

  Thereby, the facing determination based on the first to Nth peripheral regions can be performed. Specifically, as described above, the brightness distribution in a specific direction such as the left-right direction, the up-down direction, or the diagonal direction is determined. Therefore, it becomes possible to determine that the situation as shown in FIG. When the imaging direction is oblique to the in-vivo subject as shown in FIG. 8B, the first portion of the peripheral area of the in-vivo image (upper side in FIG. 8C) is a subject that is far from the imaging optical system. The second portion of the peripheral area (lower side in FIG. 8C) becomes brighter because it corresponds to a subject that is close in distance. Therefore, the comparison result of the brightness information of the first part and the second part in the peripheral region (specifically, the comparison of whether or not the difference between the values represented by the brightness information is larger than a given threshold value) Based on the result, it is possible to distinguish from the case of facing the face (FIG. 12C). In this embodiment mentioned above, it corresponds to using D2-D5 of Formula (2) for determination.

  Further, when the value represented by the brightness information calculated for each of the areas divided by the dividing unit 3213 is smaller than a given threshold, the facing determination unit 321 causes the in-vivo subject to enter the imaging optical system. It is determined that they are facing each other.

  This makes it possible to determine whether or not the face is correct by comparing the value represented by the brightness information calculated for each region (for example, the average value of brightness values) with a given threshold value. When facing each other, as shown in FIG. 12C, the brightness in the entire in-vivo image is nearly uniform, and therefore the difference in brightness between regions is small. Therefore, as an example, two regions are extracted from a plurality of regions, and the comparison process between the difference value of the value represented by the brightness information of the extracted regions and the threshold value is combined with all region extractions (in-body If the image is divided into M areas, a method of performing M × (M−1) / 2) is conceivable. However, FIG. 12A and FIG. 12B are conceivable as characteristic brightness distributions when not facing each other, so that the processing load of the comparison process can be reduced. For example, as described above, the in-vivo image is divided into a central area and a peripheral area, and a difference value is obtained from a value represented by the brightness information of the central area and a value represented by the brightness information of the peripheral area. You may perform a comparison process with this threshold value. Further, the in-vivo image is divided into a central region and first to Nth peripheral regions, and a value represented by brightness information of the i-th to j-th peripheral regions and the brightness of the k-th to m-th peripheral regions. The difference value may be obtained from the value represented by the information, and the comparison process may be performed with a given threshold value.

  Further, as shown in FIG. 5, the focal position control device includes an evaluation region setting unit 322 that sets an evaluation region for the in-vivo image, and a contrast value calculation unit 323 that calculates the contrast value of the set evaluation region. May be included. Then, the focal position control unit 325 controls the focal position of the imaging optical system based on the contrast value.

  Here, the evaluation area is an area for which a contrast value is to be calculated, and the position and size of the evaluation area may be determined by the system, or may be determined by the user from the external I / F unit 500 or the like. You may enter.

  Thereby, it is possible to control the focal position of the imaging optical system using the contrast value. Specifically, it is realized as contrast AF or the like.

  Further, the contrast value calculation unit 323 may include a bright spot removal unit 3231 that removes the bright spots included in the evaluation region as illustrated in FIG. 7. Then, the contrast value calculation unit 323 calculates the contrast value of the evaluation area from which the bright spot is removed.

  As a result, the presence of bright spots makes it possible to suppress adverse effects on focus position control. The contrast value can be calculated by applying a high-pass filter to the evaluation area, for example. That is, the higher the high frequency component, the higher the contrast value. Contrast AF originally calculates a contrast value based on a high-frequency component caused by a subject (for example, a component included in an image signal when the subject structure is clear), and a state where the contrast value is high is a focused state. And the focal position is controlled. However, if a bright spot exists, the bright spot is a point having a large difference in pixel value from the surrounding pixels, and thus a high frequency component appears regardless of the focused state of the subject. Therefore, the contrast value may increase even when the subject is not in focus, and focus position control (AF operation) cannot be performed properly. For the above reasons, it is desirable to remove the bright spots included in the evaluation area before calculating the contrast value.

  Further, in the present embodiment described above, an image acquisition unit (A in FIG. 1) acquires an in-vivo image including an image of an in-vivo subject via an imaging optical system (including the objective lens 230 and the focus adjustment lens 240 in FIG. 1). / D conversion unit 310) and a determination unit (facility determination unit 321 in FIG. 5 or stationary in FIG. 18) that determines the status information of the in-vivo subject with respect to the imaging direction of the imaging optical system at the time of imaging based on the in-vivo image. A focal position control unit 325 that controls the focal position of the imaging optical system based on the determination result in the determination unit, and a focal position that drives the imaging optical system based on the control of the focal position control unit 325. The present invention relates to an endoscope apparatus including a driving unit 250.

  Accordingly, it is possible to realize an endoscope apparatus that controls the focal position based on the status information of the in-vivo subject with respect to the imaging direction of the imaging optical system and drives the imaging optical system according to the control. The situation information may be posture information (specifically, whether or not it is directly facing) as described above, or motion information (specifically, as described later in the third and fourth embodiments). Or whether it is relatively stationary).

2. Second Embodiment FIG. 16 shows a configuration example of an endoscope apparatus including a focal position control apparatus according to the present embodiment. In the first embodiment, the focus adjustment lens 240 is driven by the focus position drive unit 250 to adjust the focus position and the angle of view. In this embodiment, as shown in FIG. 16, the focus adjustment is performed. It is assumed that the focus position is adjusted by driving the lens 240 (focus lens in a narrow sense) by the focus position driving unit 250, and the angle of view is adjusted by driving the zoom lens 270 by the zoom lens driving unit 280. However, although the present embodiment will be described using an imaging optical system driven by two lens groups, the imaging optical system may be one driven by a group of lenses.

  FIG. 17 shows a detailed configuration example of the focus lens control unit 320 in the second embodiment. The function of the focal position control unit 325 is different from that of the first embodiment. The zoom lens control unit 350 is connected to the focal position control unit 325.

  The focal position control unit 325 receives a control signal for starting the AF operation from the facing determination unit 321 and a control signal indicating that the zoom lens 270 is at the telephoto end from the zoom lens control unit 350 Controls the focus position drive unit 250 based on the focus position information transferred from the focus position detection unit 324. On the other hand, the focus position control unit 325 receives a control signal indicating that the zoom lens 270 is not at the telephoto end when the control signal for stopping the AF operation is transferred from the facing determination unit 321 or from the zoom lens control unit 350. When it is transferred, the focus position driving unit 250 is controlled so as to move the focus adjustment lens 240 to a predetermined position stored in the control unit 340 in advance.

  Although it has been described here that the zoom lens 270 is at the telephoto end is one condition for performing the focusing operation, the present invention is not limited to this and is within a given range including the telephoto end. When the zoom lens 270 is positioned, it may be determined that one condition for performing the focusing operation is satisfied. Specifically, as will be described later, the focusing operation may be performed when the zoom lens is located on the telephoto side with respect to the reference point and is directly facing.

  As described above, in this embodiment, only when the depth of field of the optical system is narrow, the AF operation is started when the imaging unit faces the subject. As a result, when the subject has a depth, observation is performed with pan focus, and a suitable image according to the scene can be obtained by performing the AF operation under the condition that the depth of field becomes narrow, such as during magnified observation. .

  In the present embodiment described above, the focal position control unit 325 is configured such that the zoom lens 270 is telephoto with respect to a reference point located between the telephoto end and the wide-angle end in the movable range of the zoom lens 270 (included in the imaging optical system). If it is determined that the in-body subject and the imaging optical system are facing each other, the focusing operation is started.

  Thereby, in addition to the determination of facing in the first embodiment, it is possible to control whether or not the focusing operation is performed in consideration of the positional relationship between the zoom lens 270 and the reference point. As a general optical characteristic, the depth of field becomes narrower as the angle of view becomes narrower (the zoom lens 270 moves to the telephoto end and the magnification becomes higher). If the depth of field is narrowed, it is difficult for the user to manually focus, so that the advantage of performing an automatic focusing operation (AF operation) by the system is increased. Conversely, when the zoom lens 270 is on the wide-angle side, the depth of field is relatively wide, so there is a high possibility that manual focusing can be performed without using AF. Also, in AF, the focus area (evaluation area for calculating the contrast value in contrast AF) does not match the area that the user wants to observe, and performing AF may rather hinder observation. In consideration, it is desirable to include not operating the AF as an option. From the above, by using the positional relationship between the zoom lens 270 and the reference point as a determination condition for focus position control, an appropriate AF operation in consideration of the zoom lens position (that is, the depth of field) is realized. It becomes possible.

3. Third Embodiment A configuration example of an endoscope apparatus including a focal position control apparatus according to the present embodiment is shown in FIG. 16 as in the second embodiment. As shown in FIG. 16, in the present embodiment, description will be made using an imaging optical system driven by two lens groups, but the imaging optical system may be one driven by a group of lenses.

  In the present embodiment, the operation of the focus lens control unit 320 is different from that of the second embodiment. FIG. 18 shows a detailed configuration example of the focus lens control unit 320 in the present embodiment. The facing determination unit 321 is removed from the second embodiment, and a stillness determination unit 326 is added. The image processing unit 330 is connected to the evaluation area setting unit 322 and the stillness determination unit 326. The stillness determination unit 326 is connected to the focal position control unit 325.

  FIG. 19 shows a detailed configuration example of the stillness determination unit 326. The stillness determination unit 326 includes a storage unit 3261, a motion detection unit 3262, and a determination unit 3263. The image processing unit 330 is connected to the storage unit 3261 and the motion detection unit 3262. The storage unit 3261 is connected to the motion detection unit 3262. The motion detection unit 3262 is connected to the determination unit 3263. The determination unit 3263 is connected to the focal position control unit 325.

  The storage unit 3261 stores the image signal transferred from the image processing unit 330. The motion detection unit 3262 detects a relative motion between the subject and the imaging unit from the image signal. Specifically, the motion detection unit 3262 matches the image signal input from the image processing unit 330 with the feature points of the image signal acquired one frame before stored in the storage unit 3261, and the matched feature points The amount of motion Mv is calculated from the motion vector. For example, the motion amount Mv is an average of motion vectors of feature points of the entire image. The calculated motion amount Mv is transferred to the determination unit 3263.

  The determination unit 3263 compares the calculated movement amount Mv with a predetermined threshold ThMv1, and determines whether or not the movement of the subject on the image is large. The threshold ThMv1 may be a preset value or may be automatically set by the control unit 340. When the movement amount Mv1 is smaller than the threshold ThMv1, that is, when the movement of the subject is small, the determination unit 3263 determines that the surgeon is observing the subject in detail. In this case, the determination unit 3263 transfers a control signal for starting the AF operation to the focal position control unit 325. On the other hand, the determination unit 3263 determines that the surgeon is screening the subject when the amount of motion Mv1 is greater than or equal to the threshold ThMv1, that is, when the subject's motion is large. In this case, the determination unit 3263 transfers a control signal for stopping the AF operation to the focus position control unit 325.

  The focus position control unit 325 receives a control signal for starting an AF operation from the stillness determination unit 326 and a control signal indicating that the zoom lens 270 is at the telephoto end from the zoom lens control unit 350. Based on the focus position information transferred from the focus position detection unit 324, the focus position drive unit 250 is controlled. On the other hand, the focus position control unit 325 transfers a control signal indicating that the AF operation is stopped from the stillness determination unit 326 or a control signal indicating that the zoom lens 270 is not at the telephoto end from the zoom lens control unit 350. If it is, the focus position driving unit 250 is controlled to move the focus adjustment lens 240 to a predetermined position stored in the control unit 340 in advance.

  As described above, in the present embodiment, it is determined whether or not the imaging unit and the subject are relatively stationary from the image signal, and when it is determined that the imaging unit is stationary with respect to the subject, The AF operation is started. On the other hand, when it is determined that the imaging unit is not stationary with respect to the subject, the AF operation is stopped. Thereby, the surgeon does not need to perform the start / stop operation of the AF operation, and can obtain a suitable image according to the scene.

  In the above description, the stillness determination unit 326 determines whether or not it is still, and does not consider the direction of movement. However, the relative movement between the imaging unit 200 and the subject intersects with the first movement in the direction along the imaging direction (optical axis direction) of the imaging unit 200 and intersects the imaging direction (in the narrow sense, orthogonal). ) Second movement in direction is conceivable. When there is a first movement, the relative distance between the imaging unit 200 and the subject fluctuates, which increases the possibility that the subject will deviate from the depth of field. Also, the subject being imaged itself does not change significantly. Therefore, in this case, it is conceivable that AF is effective even when it is determined that the amount of motion is large and not stationary. That is, for the second movement, the AF may be operated when the amount of movement is small (still) as described above, but the first movement is performed differently from the above method. There is a good case. From the above, as a modification of the present embodiment, as a method of using a motion amount used for determination in the stillness determination unit 326 that excludes the motion amount due to the first motion and uses the motion amount due to the second motion. explain.

  When calculating the motion vector described above, the motion detection unit 3262 sets representative points at regular intervals and in a grid pattern with respect to the pixels of the current frame image (the image output from the image processing unit 330). In FIG. 21, representative points are indicated by black circles. Then, local motion vectors of the current frame image and the previous frame image (image stored in the storage unit 3261) at the representative point are detected using block matching or the like that is a known technique. The coordinates of the set representative point and the motion vector detected at each representative point are output to the determination unit 3263.

  Next, the determination unit 3263 determines whether the motion represented by the motion vector is the first motion or the second motion. A specific determination method will be described with reference to FIGS. 22 (A) to 22 (D). First, the vanishing point of the motion vector is detected based on the motion vector at the representative point.

  In FIG. 22A, a black circle indicates a representative point, a solid line arrow indicates a motion vector, and a cross indicates a vanishing point. The vanishing point is an intersection of straight lines extending from each representative point in the direction of the motion vector at each representative point. When the observation target is a cylindrical shape having a constant inner diameter, and the motion vector is generated only by the movement of the imaging unit 200, the straight lines intersect at one point at the vanishing point during the first movement. Actually, the inner diameter of the luminal organ to be observed is not constant, and the motion vector is also generated by the pulsation of the living body. Therefore, even if the first motion occurs, the straight line is one point at the vanishing point. Do not cross. Therefore, the sum of the distances from all the straight lines is set as the first evaluation value, and the point with the smallest first evaluation value is first set as the vanishing point candidate.

  Next, when the first evaluation value in the vanishing point candidate is equal to or smaller than the predetermined first threshold value, and when the vanishing point candidate exists in the image, the vanishing point candidate is set as the vanishing point. Determines that the vanishing point cannot be detected. Here, for the sake of explanation, the direction of movement of the endoscope tip is defined as shown in FIG. The x, y, and z directions correspond to the horizontal direction, the vertical direction, the horizontal direction, and the depth direction orthogonal to the vertical direction, respectively, on the image. Further, in each of the x, y, and z directions, the movement along the arrow is defined as a positive movement, and the opposite movement is defined as a negative movement. When there is a first movement, the tip of the imaging unit 200 shows a negative movement in the z direction. When the movement is not the first movement (for example, in the case of the second movement), for example, the movement of the tip of the imaging unit 200 is a positive movement in the x direction, and the motion vectors are substantially parallel as shown in FIG. Therefore, even if the vanishing point candidate exists outside the image or exists in the image, the first evaluation value in the vanishing point candidate is a large value. Therefore, by detecting the presence of the vanishing point based on the position of the vanishing point candidate and the first evaluation value, the movement in the z direction can be detected. However, as shown in FIG. 22D, since the vanishing point can be detected even when an endoscope is inserted, the vector from the representative point to the vanishing point (broken arrow) and the motion vector (solid arrow) Based on the determination. Specifically, if the number of representative points at which the inner product of the vector to the vanishing point and the motion vector is negative is greater than or equal to a predetermined number, it is determined that the vanishing point cannot be detected. When the vanishing point cannot be detected, it is determined that the movement at that time is not the first movement.

  In the present embodiment described above, the determination unit performs determination using information regarding whether or not the in-vivo subject is stationary relative to the imaging optical system as the motion information described above in the first embodiment. The determination unit 326 may be included. The focal position control unit 325 controls the focal position of the imaging optical system based on the determination result in the stillness determination unit 326.

  Here, that the in-vivo subject is stationary relative to the imaging optical system is not limited to a state in which there is no relative movement, and the index value representing the movement is less than a given threshold value. This includes cases where there is some small movement. As the index value representing the movement, for example, it is conceivable that the amount of movement is used as described above.

  This makes it possible to control the focal position of the imaging optical system based on whether or not it is determined that the in-vivo subject is relatively stationary with respect to the imaging optical system. The case where it is not stationary is assumed to be a situation such as screening in an endoscope apparatus, for example. In this case, since it is sufficient that the zoom magnification is low, the depth of field is widened and the advantage of performing the focus position control (particularly the AF operation) is not great. In addition, since the subject to be imaged changes greatly in a short time, the focus position control is performed each time, which is not efficient. Therefore, it is possible to perform appropriate focal position control according to the situation by determining whether or not the camera is stationary.

  The focus position control unit 325 performs an operation for focusing on the in-vivo subject (in-focus) when the in-vivo subject is determined to be relatively stationary with respect to the imaging optical system in the stationary determination unit 326. Operation) may be started. When it is determined that the in-vivo subject is not relatively stationary, the focusing operation is stopped, and an imaging optical system (specifically, a zoom lens 270 included in the imaging optical system) is given. It may be moved to the position.

  Here, the given position to which the imaging optical system is moved when not stationary is the same as the case where the imaging optical system is not directly facing in the first embodiment, and thus detailed description thereof is omitted.

  This makes it possible to control the focusing operation on the in-vivo subject being imaged (that is, corresponding to AF control) depending on whether or not the camera is stationary. As described above, AF is not effective when not stationary. In other words, by controlling the AF operation based on the determination of whether or not the camera is stationary, it is possible to operate the AF when the AF is effective, and not to operate the AF when the AF is not effective. The user's observation (such as diagnosis using an endoscope apparatus) can be made smooth.

  Stillness determination unit 326 may include a storage unit 3261 and a motion detection unit 3262 as shown in FIG. The storage unit 3261 stores the first in-vivo image acquired at the first timing and the second in-vivo image acquired at the second timing different from the first timing. The motion detection unit 3262 detects a relative motion amount between the in-vivo subject and the imaging optical system based on the first in-vivo image and the second in-vivo image. The stillness determination unit 326 determines that the in-vivo subject is stationary relative to the imaging optical system when the amount of motion is smaller than a given threshold.

  Here, the motion amount represents the degree of motion of the in-vivo subject between the two in-vivo images, and is, for example, an average value of the motion vectors obtained from the first in-vivo image and the second in-vivo image. May be. The motion vector is obtained by, for example, matching processing.

  Thereby, the stillness determination based on the amount of movement becomes possible. Since the amount of motion can be obtained from a relatively simple algorithm such as matching processing as described above, stillness determination based on the amount of motion can be easily realized.

  Further, the first movement, which is a relative movement of the in-vivo subject with respect to the imaging optical system, along the imaging direction (optical axis direction) of the imaging optical system intersects with the imaging direction (optical axis direction) of the imaging optical system. Consider a second movement that is a relative movement of the in-vivo subject with respect to the imaging optical system along the direction. In this case, the stillness determination unit 326 determines whether the in-vivo subject is stationary relative to the imaging optical system based on the second movement of the first movement and the second movement. May be performed.

  Here, the first movement is a movement in the optical axis direction of the imaging optical system in a narrow sense, but includes a movement in which an angle formed with the optical axis direction is equal to or less than a given angle threshold. Similarly, the second movement is a movement in a direction orthogonal to the optical axis direction of the imaging optical system in a narrow sense, but the angle formed with the direction orthogonal to the optical axis is equal to or less than a given angle threshold value. It includes the movement that becomes.

  Accordingly, when there is a first movement and a second movement, it is possible to perform stillness determination based on the second movement. When there is a second movement (the amount of movement due to the second movement is larger than the threshold), the subject to be imaged changes greatly as described above, or screening by an endoscope apparatus is assumed. Even if it is used for stillness determination as described in the present embodiment (specifically, AF is operated when it is stationary, and AF is not operated when it is not stationary, etc.) No problem) However, in the first movement, the subject to be imaged does not change greatly, and the size of the subject in the in-vivo image changes. Therefore, when there is a first movement, it is assumed that the user desires continuous observation of the same subject. Furthermore, since the distance between the in-vivo subject and the imaging optical system changes due to movement, there is a high possibility that the in-vivo subject will be out of the range of depth of field and blurred. That is, even when it is determined that the first movement is large (not stationary), it is desirable to positively control the focal position (for example, activate AF). For the above reasons, in the stillness determination of the present embodiment executed by the stillness determination unit 326, it is preferable to use the second movement of the first movement and the second movement.

  Stillness determination section 326 performs vanishing point detection processing based on a plurality of motion vectors at a plurality of representative points set on the in-vivo image, and the in-vivo subject is imaged optically based on the vanishing point detection result. It may be determined whether or not the system is stationary relative to the system.

  Accordingly, it is possible to perform stillness determination by performing vanishing point detection processing on the in-vivo image. Since the direction of relative movement between the in-vivo subject and the imaging optical system can be estimated based on the presence or absence of the vanishing point, stillness determination can be performed in consideration of the estimated relative movement direction. Specifically, by the vanishing point detection process, it is determined whether the relative movement between the in-vivo subject and the imaging optical system is a movement belonging to the first movement or a movement belonging to the second movement, When it is determined that the movement belongs to the second movement, the stillness determination may be performed. For example, as shown in FIGS. 22 (A) and 22 (D), the movement when the vanishing point is detected is determined as the second movement, and as shown in FIG. 22 (C), the vanishing point is determined. The movement when no is detected is determined as the first movement. The vanishing point detection process, as described above, when the point with the smallest sum of the distance to the straight line obtained by extending the motion vector is the vanishing point candidate, the vanishing point candidate exists in the in-vivo image, and You may perform by determination whether a 1st evaluation value is below a threshold value. At this time, as the first evaluation value, the total sum of the distances from the straight line obtained by extending the motion vector may be used as it is.

  In addition, the focal position control unit 325 is configured such that the zoom lens 270 is positioned on the telephoto side with respect to a reference point located between the telephoto end and the wide-angle end in the movable range of the zoom lens 270 (included in the imaging optical system). If the in-vivo subject and the imaging optical system are determined to be relatively stationary in the stationary determination unit 326, the focusing operation may be started.

  Accordingly, it is possible to control whether or not the focusing operation is performed in consideration of the positional relationship between the zoom lens 270 and the reference point in addition to the stillness determination. The advantages of considering the zoom lens position are the same as in the case of the second embodiment, and a detailed description thereof will be omitted.

4). Fourth Embodiment A configuration example of an endoscope apparatus including a focal position control apparatus according to the present embodiment is shown in FIG. 16 as in the second embodiment. As shown in FIG. 16, in the present embodiment, description will be made using an imaging optical system driven by two lens groups, but the imaging optical system may be one driven by a group of lenses.

  In the present embodiment, the operation of the focus lens control unit 320 is different from that of the second embodiment. FIG. 20 shows a detailed configuration example of the focus lens control unit 320 in the fourth embodiment. A stillness determination unit 326 is added to the second embodiment. The image processing unit 330 is connected to a facing determination unit 321, an evaluation area setting unit 322, and a stillness determination unit 326. The stillness determination unit 326 is connected to the focal position control unit 325.

  The configuration and operation of the stillness determination unit 326 are the same as those in the third embodiment. The focus position control unit 325 receives the control signal for starting the AF operation from the facing determination unit 321, transfers the control signal for starting the AF operation from the stationary determination unit 326, and receives the control signal from the zoom lens control unit 350. When a control signal indicating that the zoom lens 240 is at the telephoto end is transferred, the focus position driving unit 250 is controlled based on the focus position information transferred from the focus position detection unit 324. On the other hand, the focus position control unit 325 receives a control signal for stopping the AF operation from the facing determination unit 321, or receives a control signal for stopping the AF operation from the stationary determination unit 326, or When a control signal indicating that the zoom lens 270 is not at the telephoto end is transferred from the zoom lens control unit 350, the focus position is adjusted so that the focus adjustment lens 240 is moved to a predetermined position stored in advance in the control unit 340. The drive unit 250 is controlled.

  In the above-described embodiment, the determination unit determines whether the in-vivo subject is directly facing the imaging optical system based on the in-vivo image, and based on the in-vivo image, based on the confrontation determination unit 321 that determines the posture information. A stationary determination unit 326 that determines whether the in-vivo subject is stationary relative to the imaging optical system as motion information is included. Then, the focus position control unit 325 starts the focusing operation when it is determined that the facing determination unit 321 faces the camera and the stationary determination unit 326 determines that the camera is stationary.

  As a result, the focus operation is started (the AF is operated) after performing both the determination of the confrontation described in the first embodiment and the determination of the stillness described in the third embodiment. It becomes possible. Therefore, in a situation where AF is not effective or AF is unnecessary, it is possible to prevent AF from being operated, and it is possible to provide an image that is easy for the user to view.

  In addition, the focal position control unit 325 is configured such that the zoom lens 270 is positioned on the telephoto side with respect to a reference point located between the telephoto end and the wide-angle end in the movable range of the zoom lens 270 (included in the imaging optical system). Further, when it is determined that the in-vivo subject and the imaging optical system are directly facing in the facing determination unit 321 and it is determined in the stationary determination unit 326 that the in-vivo subject and the imaging optical system are relatively stationary. May start the focusing operation.

  Accordingly, it is possible to control whether or not the focusing operation is performed in consideration of the positional relationship between the zoom lens 270 and the reference point, in addition to the determination of the confrontation and the stillness. The advantages of considering the zoom lens position are the same as in the case of the second embodiment, and a detailed description thereof will be omitted.

  As mentioned above, although four Embodiments 1-4 and its modification to which this invention was applied were demonstrated, this invention is not limited to each Embodiment 1-4 and its modification as it is, An implementation step The constituent elements can be modified and embodied without departing from the spirit of the invention. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described first to fourth embodiments and modified examples. For example, you may delete a some component from all the components described in each Embodiment 1-4 and the modification. Furthermore, you may combine suitably the component demonstrated in different embodiment and modification. Thus, various modifications and applications are possible without departing from the spirit of the invention.

100 light source unit, 110 white light source, 120 rotation color filter, 130 rotation drive unit,
140 condenser lens, 200 imaging unit, 210 light guide fiber,
220 illumination lens, 230 objective lens, 240 focus adjustment lens,
250 focus position drive unit, 260 imaging element, 270 zoom lens,
280 Zoom lens driving unit, 300 control device, 310 A / D conversion unit,
320 focus lens control unit, 321 alignment determination unit, 322 evaluation region setting unit,
323 contrast value calculation unit, 324 focus position detection unit,
325 focus position control unit, 326 stillness determination unit, 330 image processing unit,
331 pre-processing unit, 332 synchronization unit, 333 post-processing unit, 340 control unit,
350 zoom lens control unit, 400 display unit, 500 external I / F unit,
3211 correction unit, 3212 low-frequency extraction unit, 3213 division unit,
3214 Brightness calculation unit, 3215 comparison unit, 3216 determination unit,
3231 bright spot removal unit, 3232 high-frequency extraction unit, 3261 storage unit,
3262 detection unit, 3263 determination unit

Claims (25)

An image acquisition unit that acquires an in-vivo image including an image of the in-vivo subject by imaging an in-vivo subject via an imaging optical system;
A determination unit that determines state information of the in-vivo subject with respect to an imaging direction of the imaging optical system at the time of imaging based on the acquired in-vivo image;
A focal position control unit that controls the focal position of the imaging optical system by controlling the driving of the imaging optical system based on the determination result in the determination unit;
A focal position control device comprising: In claim 1,
The determination unit
A focus position control apparatus that determines at least one of posture information of the in-vivo subject with respect to the imaging direction of the imaging optical system and movement information of the in-vivo subject with respect to the imaging direction as the status information of the in-vivo subject. . In claim 2,
The determination unit
A face-to-face determination unit that determines, as the posture information, whether or not the in-vivo subject faces the imaging optical system based on the in-vivo image;
The focal position controller is
A focal position control device that controls the focal position of the imaging optical system based on a determination result in the confrontation determination unit. In claim 3,
The focal position controller is
The focal position, wherein when the in-body determination unit determines that the in-vivo subject is directly facing the imaging optical system, a focusing operation for starting focusing on the in-vivo subject is started. Control device. In claim 4,
The focal position controller is
When it is determined by the facing determination unit that the in-vivo subject is not facing the imaging optical system, the focusing operation is stopped, and the zoom lens included in the imaging optical system is moved to a given position. A focal position control device characterized by being moved to a position. In claim 3,
The focal position controller is
The zoom lens included in the imaging optical system is on the telephoto side with respect to a reference point located between the telephoto end and the wide-angle end, and the in-body determination unit corrects the in-vivo subject to the imaging optical system. When it is determined that the subject is in focus, the focus position control device starts a focusing operation for focusing on the in-vivo subject. In claim 3,
The facing determination unit
A brightness calculation unit that calculates brightness information based on an image signal of the in-vivo image;
The facing determination unit
A focal position control device that determines whether or not the in-vivo subject is directly facing the imaging optical system based on the calculated brightness information. In claim 7,
The facing determination unit
A dividing unit that divides the in-vivo image into a plurality of regions;
The brightness calculation unit
Calculating the brightness information for each of the divided areas;
The facing determination unit
A focal position control apparatus that determines whether or not the in-vivo subject is directly facing the imaging optical system based on the brightness information calculated for each region. In claim 8,
The dividing unit is
Dividing the in-vivo image into a central region and at least one peripheral region;
The facing determination unit
The focus is characterized by determining whether or not the in-vivo subject is facing the imaging optical system by performing a comparison process of the brightness information of the central region and the brightness information of the peripheral region. Position control device. In claim 8,
The dividing unit is
The in-vivo image is divided into a central region and first to Nth (N is an integer of 2 or more) peripheral regions,
The facing determination unit
The brightness information in the i-th (i is an integer of 1 ≦ i <N) to j-th (j is an integer of i ≦ j <N) of the first to N-th peripheral regions, Comparison processing of the brightness information in the k-th (m is an integer of k ≦ m ≦ N) to the m-th (m is an integer of k ≦ m ≦ N) of the first to N-th peripheral regions. A focus position control device characterized by determining whether or not the in-vivo subject is directly facing the imaging optical system. In claim 8,
The facing determination unit
When the difference between the values represented by the brightness information calculated for each region is smaller than a given threshold, it is determined that the in-vivo subject is directly facing the imaging optical system. Focus position control device. In claim 2,
The determination unit
A stationary determination unit that determines, as the motion information, whether or not the in-vivo subject is stationary relative to the imaging optical system based on the in-vivo image;
The focal position controller is
A focus position control device that controls the focus position of the imaging optical system based on a determination result in the stillness determination unit. In claim 12,
The focal position controller is
When the in-vivo subject is determined to be relatively stationary with respect to the imaging optical system, the stationary determination unit starts an in-focus operation for focusing on the in-vivo subject. Focus position control device. In claim 13,
The focal position controller is
When the in-vivo subject is determined not to be relatively stationary with respect to the imaging optical system, the stationary determination unit stops the focusing operation, and uses a zoom lens included in the imaging optical system. A focal position control device characterized by moving to a given position. In claim 12,
The stationary determination unit
A storage unit for storing a first in-vivo image acquired at a first timing and a second in-vivo image acquired at a second timing different from the first timing;
A motion detection unit that detects a relative motion amount of the in-vivo subject and the imaging optical system based on the first in-vivo image and the second in-vivo image;
The stationary determination unit
A focus position control device, wherein when the detected amount of motion is smaller than a given threshold, it is determined that the in-vivo subject is stationary relative to the imaging optical system. In claim 15,
The stationary determination unit
The first relative movement of the in-vivo subject along the imaging direction of the imaging optical system and the second relative movement of the in-vivo subject along the direction intersecting the imaging direction of the imaging optical system A focal position control device that determines whether the in-vivo subject is stationary relative to the imaging optical system based on the second movement. In claim 16,
The stationary determination unit
Based on a plurality of motion vectors at a plurality of representative points set on the in-vivo image, vanishing point detection processing is performed, and the in-vivo subject is detected with respect to the imaging optical system based on the vanishing point detection result. A focal position control device, characterized by determining whether or not it is relatively stationary. In claim 12,
The focal position controller is
The zoom lens included in the imaging optical system is on the telephoto side with respect to a reference point located between the telephoto end and the wide-angle end, and the in-vivo subject is located relative to the imaging optical system in the stationary determination unit. A focus position control apparatus characterized by starting a focusing operation for focusing on the in-vivo subject when it is determined that the subject is relatively stationary. In claim 2,
The determination unit
A facing determination unit that determines, as the posture information, whether or not the in-vivo subject is facing the imaging optical system based on the in-vivo image;
A stationary determination unit that determines, as the motion information, whether or not the in-vivo subject is stationary relative to the imaging optical system based on the in-vivo image;
The focal position controller is
In the facing determination unit, it is determined that the in-vivo subject is facing the imaging optical system, and in the stationary determination unit, the in-vivo subject is relatively stationary with respect to the imaging optical system. When it is determined that the focus position control apparatus starts focusing operation for focusing on the in-vivo subject. In claim 19,
The focal position controller is
The zoom lens included in the imaging optical system is on the telephoto side with respect to a reference point located between the telephoto end and the wide-angle end, and the in-body determination unit corrects the in-vivo subject to the imaging optical system. And when the in-vivo subject is determined to be relatively stationary with respect to the imaging optical system, the focusing operation is started. A focus position control device. In claim 1,
An evaluation area setting unit for setting an evaluation area for the in-vivo image;
A contrast value calculation unit for calculating a contrast value of the set evaluation area;
The focal position controller is
A focal position control apparatus that controls the focal position of the imaging optical system based on the contrast value. In claim 21,
The contrast value calculation unit
Including a bright spot removing unit for removing bright spots contained in the evaluation region,
The contrast value calculation unit
The focus position control apparatus, wherein the contrast value of the evaluation area from which the bright spot is removed is calculated. An image acquisition unit that acquires an in-vivo image including an image of the in-vivo subject by imaging an in-vivo subject via an imaging optical system;
A determination unit that determines state information of the in-vivo subject with respect to an imaging direction of the imaging optical system at the time of imaging based on the acquired in-vivo image;
A focal position control unit that controls the focal position of the imaging optical system by controlling the driving of the imaging optical system based on the determination result in the determination unit;
A focal position drive unit that drives the imaging optical system based on the control of the focal position control unit;
An endoscopic device comprising: By capturing an in-vivo subject via an imaging optical system, an in-vivo image including the in-vivo subject image is obtained,
Based on the acquired in-vivo image, determine the status information of the in-vivo subject relative to the imaging direction of the imaging optical system at the time of imaging,
A focus position control method, wherein the focus position of the imaging optical system is controlled by controlling driving of the imaging optical system based on a determination result. In claim 24,
At least one of posture information of the in-vivo subject with respect to the imaging direction of the imaging optical system and movement information of the in-vivo subject with respect to the imaging direction is determined as the situation information of the in-vivo subject. .