JP5547975B2 - Imaging apparatus, microscope system, and white balance adjustment method - Google Patents

Description

  The present invention relates to an image processing technique, and more particularly to a technique for improving the image quality of a captured image.

  Digital cameras and digital video cameras that are widely used nowadays have a white balance adjustment function that adjusts the color signal of a captured color image so that the color signal becomes a certain ratio. Many of them have an auto white balance function in which the camera itself performs this white balance adjustment so that the user can easily take a high-quality image.

  In auto white balance, the color signals of a subject are integrated for each color to obtain a correlation between the colors, and the balance of the color components is corrected so that the color components are at the same level with respect to changes in the light source or the like. However, when the color signals other than the achromatic region of the subject are integrated, or when noise such as whiteout is mixed in the integrated color signal, the white balance cannot be adjusted appropriately. Therefore, a technique for selecting a color signal other than the achromatic area of the subject and a technique for suppressing the influence of noise have been proposed.

  For example, in Patent Document 1, a high-frequency component amount of a luminance signal in a video signal is detected, and white balance adjustment is performed using a color signal corresponding to the luminance signal when the high-frequency component amount is a predetermined threshold value or less. Technology is disclosed.

  Further, for example, Patent Document 2 does not include a pixel including noise by adjusting the lower limit value of the luminance value so that the number of pixels whose luminance value is within a predetermined range falls within the predetermined range. A technique for performing white balance adjustment by selecting a color image signal is disclosed.

JP-A-5-75919 JP 2001-313952 A

  The technique of Patent Document 1 is based on the amount of high-frequency components based on the assumption that a region containing a lot of high-frequency components in a captured image is a subject portion, and the other portions are achromatic regions and are background portions. An achromatic region is extracted. However, in reality, there are cases in which the captured image is a portion that does not contain many high-frequency components but is a subject portion. In such a case, the white balance cannot be adjusted appropriately.

  In the technique of Patent Document 2, for example, in an image of an observation image of a specimen by a microscope, even a thin part of the specimen is extracted as an achromatic region depending on the setting of the allowable range of luminance values and the allowable range of the number of pixels. May end up. In such a case, the white balance cannot be adjusted appropriately.

  The present invention has been made in view of the above-described problems, and a problem to be solved is to improve the accuracy of auto white balance adjustment.

An imaging apparatus according to one aspect of the present invention includes an imaging device that images a subject, and imaging means that generates a captured image of the subject in which colors are expressed for each pixel by image signals of a plurality of color components. Area dividing means for dividing the captured image into a plurality of small areas, and the standard deviation of the image signal of the pixel for each color component for the pixels included in each of the plurality of small areas in the captured image Standard deviation calculating means for calculating each area, and an average value for calculating the average value of the image signal of the pixel for each color component for each pixel included in each of the plurality of small areas in the captured image The captured image is based on a noise model that is stored in advance in a storage unit and indicates a correspondence relationship between the magnitude of the image signal and the amount of noise generated by the imaging element superimposed on the image signal. The compound in Among the small regions, a small region in which the standard deviation with respect to the average value is within a predetermined range with reference to the amount of noise when the average value is regarded as the size of the image signal in the noise model is defined as a flat region. Based on the white balance gain of the flat area calculated by the flat area extracting means for extracting, the white balance gain calculating means for calculating the white balance gain of the flat area in the captured image, and the white balance gain of the flat area calculated by the white balance gain calculating means White balance adjustment means for adjusting the white balance of the captured image generated by the means, and among the small areas extracted by the flat area extraction means as flat areas, all of the average values of the image signals for each color component are the maximum values. Or the average value is greater than or equal to the threshold set on the basis of the maximum average value for each color component. White balance gain calculation optimal region extraction means for extracting a small region as a white balance gain calculation optimal region, and the white balance gain calculation means includes a white balance of the white balance gain calculation optimal region in the captured image. The gain is calculated, and the white balance adjustment unit adjusts the white balance of the captured image generated by the imaging unit based on the white balance gain of the white balance gain calculation optimum region calculated by the white balance gain calculation unit. The above-described problems are solved by this feature.

An imaging device according to another aspect of the present invention includes an imaging device that captures an image of a subject. A captured image of the subject in which colors are expressed for each pixel by image signals of a plurality of color components. A standard of image signals of pixels for each color component for imaging means to be generated, area dividing means for dividing the captured image into a plurality of small areas, and pixels included in each of the plurality of small areas in the captured image Standard deviation calculating means for calculating a deviation for each small area, and an average value of the image signal of the pixel for each color component for each pixel included in each of the plurality of small areas in the captured image, for each small area Based on an average value calculating means to be calculated and a noise model that is stored in advance in the storage unit and indicates a correspondence relationship between the magnitude of the image signal and the amount of noise generated by the imaging device superimposed on the image signal , In the captured image Among the plurality of small regions, a small region in which the standard deviation with respect to the average value is within a predetermined range based on a noise amount when the average value is regarded as the size of the image signal in the noise model, Based on the flat area extraction means for extracting as a flat area, the white balance gain calculation means for calculating the white balance gain of the flat area in the captured image, and the white balance gain of the flat area calculated by the white balance gain calculation means White balance adjusting means for adjusting white balance of a captured image generated by the imaging means, and flat area extraction determining means for determining whether or not the flat area extracting means has extracted a small area that is the flat area. When the flat area extraction means cannot extract the small area which is the flat area, the flat area extraction determination means When was boss, the re-extraction of the small region which is the flat area by the flat region extracting means, a flat area reextracted means for causing relative to another of the captured image for the subject is image pickup means for generating a It is characterized by having .

  At this time, the flat area re-extraction means determines whether or not the number of times of re-extraction of the small area, which is the flat area, performed by the flat area extraction means exceeds a predetermined number. A determination unit, and a warning unit that outputs an auto white balance failure warning when the flat region re-extraction number determination unit determines that the number of times of re-extraction of the small region that is the flat region is equal to or greater than a predetermined number of times, It can also comprise so that it may have further.

  Further, in the imaging apparatus described above, a white balance gain calculation optimal region extraction determination unit that determines whether or not the white balance gain calculation optimal region extraction unit has extracted a small region that is the white balance gain calculation optimal region; When the white balance gain calculation optimum region extraction means determines that the white balance gain calculation optimum region extraction means cannot extract the small region that is the white balance gain calculation optimum region, the white balance gain calculation optimum region A white balance gain calculation optimal region re-extraction unit that causes re-extraction of the small region, which is the white balance gain calculation optimal region by the extraction unit, to another captured image of the subject generated by the imaging unit; Furthermore, it can also comprise so that it may have.

  At this time, the number of times of re-extraction of the small region, which is the white balance gain calculation optimal region, which the white balance gain calculation optimal region re-extraction unit has performed by the white balance gain calculation optimal region extraction unit exceeds a predetermined number of times. White balance gain calculation optimal region re-extraction number determination means for determining whether or not the white balance gain calculation optimal region, and when the number of times of re-extraction of the small region, which is the white balance gain calculation optimal region, exceeds a predetermined number, the white balance gain calculation optimal region A warning means for outputting a warning indicating an auto white balance failure when the re-extraction number determination means makes a determination may be further included.

An imaging device according to another aspect of the present invention includes an imaging device that captures an image of a subject. A captured image of the subject in which colors are expressed for each pixel by image signals of a plurality of color components. A standard of image signals of pixels for each color component for imaging means to be generated, area dividing means for dividing the captured image into a plurality of small areas, and pixels included in each of the plurality of small areas in the captured image Standard deviation calculating means for calculating a deviation for each small area, and an average value of the image signal of the pixel for each color component for each pixel included in each of the plurality of small areas in the captured image, for each small area Based on an average value calculating means to be calculated and a noise model that is stored in advance in the storage unit and indicates a correspondence relationship between the magnitude of the image signal and the amount of noise generated by the imaging device superimposed on the image signal , In the captured image Among the plurality of small regions, a small region in which the standard deviation with respect to the average value is within a predetermined range based on a noise amount when the average value is regarded as the size of the image signal in the noise model, Based on the flat area extraction means for extracting as a flat area, the white balance gain calculation means for calculating the white balance gain of the flat area in the captured image, and the white balance gain of the flat area calculated by the white balance gain calculation means A white balance adjusting unit that adjusts a white balance of a captured image generated by the imaging unit, and the imaging unit captures a microscope image of an observation specimen obtained by a microscope with the imaging element, Generate a captured image of the observation specimen, and obtain information on the observation conditions of the microscope when capturing a microscope image of the observation specimen Observation condition acquisition means, the observation condition information includes information on the observation magnification of the microscope, and the region dividing means is included in the observation conditions acquired by the observation condition acquisition means. in the observation magnification of the size of the small region which has been changed in accordance with the information of who is, divides the captured image, it is characterized in.

  In addition, a microscope system according to another aspect of the present invention includes a microscope that obtains a microscope image of an observation specimen, and an imaging device that captures the microscope image obtained by the microscope, and an image of a plurality of color components. An imaging unit that generates a captured image of the observation specimen in which a color is expressed for each pixel by a signal; an area dividing unit that divides the captured image into a plurality of small regions; and a plurality of the small regions in the captured image. Standard deviation calculating means for calculating the standard deviation of the image signal of the pixel for each color component for each pixel included in each of the small regions, and the pixels included in each of the plurality of small regions in the captured image Average value calculating means for calculating the average value of the image signal of the pixel for each color component for each of the small regions, and stored in the storage unit in advance, and the size of the image signal and the image signal superimposed on the image signal The image sensor Based on a noise model indicating a correspondence relationship with the amount of noise to be generated, the standard deviation of the plurality of small regions in the captured image with respect to the average value is the magnitude of the image signal in the noise model. A flat area extracting means for extracting a small area within a predetermined range with reference to the amount of noise when viewed as a flat area, and a white balance gain calculating means for calculating a white balance gain of the flat area in the captured image; Based on the white balance gain of the flat region calculated by the white balance gain calculating means, white balance adjusting means for adjusting the white balance of the captured image generated by the imaging means, and taking a microscope image of the observation specimen Observation condition acquisition means for acquiring information on observation conditions of the microscope at the time, and The information includes the information of the observation magnification of the microscope, and the region dividing means includes the small region changed according to the information of the observation magnification included in the observation condition acquired by the observation condition acquisition means. The captured image is divided according to size, and the above-described problems are solved by this feature.

  According to the present invention, by performing the above, there is an effect of enabling automatic white balance adjustment with high accuracy.

1 is a configuration diagram of an imaging apparatus according to a first embodiment of the present invention. It is a graph which shows the example of a noise model. It is a figure which shows the example of a captured image. It is a figure showing the 1st example of a mode that the captured image of FIG. 3 was divided | segmented into the small area | region. It is a figure which shows the example of extraction of a WB gain calculation area | region. It is a flowchart showing the processing content of the auto balance process which concerns on 1st embodiment of this invention. It is a block diagram of the imaging device which concerns on 2nd embodiment of this invention. It is a figure which shows the example of extraction of WB gain calculation optimal area | region. It is an example of the graph showing the image signal average value of R color of each small area. It is an example of the graph showing the image signal average value of G color of each small area. It is an example of the graph showing the image signal average value of B color of each small area. It is a flowchart showing the processing content of the auto balance process which concerns on 2nd embodiment of this invention. It is a block diagram of the imaging device which concerns on 3rd and 4th embodiment of this invention. It is a flowchart showing the processing content of the auto balance process which concerns on 3rd embodiment of this invention. It is a flowchart showing the processing content of the auto balance process which concerns on 4th embodiment of this invention. It is a figure showing the 2nd example of a mode that the captured image of FIG. 3 was divided | segmented into the small area. It is a figure which shows a mode that the calculation object range of a standard deviation and an average value was selected from the captured image of FIG. It is a figure which shows the example of the captured image using the high magnification objective lens. It is a block diagram of the microscope system containing the imaging device which concerns on 5th embodiment of this invention. It is a flowchart showing the processing content of the auto balance process which concerns on 5th embodiment of this invention. It is the figure showing a mode that the captured image of FIG. 18 was divided | segmented into the small area.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  [First embodiment]

  FIG. 1 is a configuration diagram of an imaging apparatus according to the present embodiment. The imaging apparatus includes an imaging unit 1, a control unit 2, a white balance adjustment unit (hereinafter referred to as “WB adjustment unit”) 5, an image processing unit 6, an operation unit 7, and a display unit 8. Here, the imaging unit 1, the control unit 2, the WB adjustment unit 5, and the image processing unit 6 are all connected to the data bus 9, and exchange various data under the control of the control unit 2. Can do.

  The imaging unit 1 includes an imaging device 11, an imaging device driving unit 12, a preprocessing unit 13, an amplification unit 14, and an A / D conversion unit 15.

  The image sensor 11 is, for example, a CCD (Charge Coupled Device), and pixels that detect the luminance of each of the three primary colors of red (R), green (G), and blue (B) light are arranged in a Bayer array ( A signal corresponding to the amount of light received in each pixel unit is output. The image sensor 11 is driven with an exposure time based on the drive signal from the image sensor drive unit 12, and the output signal is input to the pre-processing unit 13.

  The pre-processing unit 13 outputs an output signal from the image sensor 11 to the amplifying unit 14 as an image signal of each pixel constituting the captured image according to the control pulse given from the image sensor driving unit 12.

  The amplification unit 14 amplifies the image signal output from the pre-processing unit 13 with a gain set by the control unit 2.

  The A / D conversion unit 15 performs analog-to-digital conversion on the image signal, which is an analog signal, amplified by the amplification unit 14 in accordance with the clock signal from the image sensor driving unit 12, and expresses a captured image. Is output. A captured image, which is digital data, output from the A / D conversion unit 15 is input to the WB adjustment unit 5 and also to the control unit 2 via the data bus 9. In the control unit 2, this captured image is stored in the memory 3.

  The imaging unit 1 is configured as described above, images a subject, and generates a captured image of the subject in which colors are expressed for each pixel by image signals of a plurality of color components.

  The imaging unit 1 sequentially captures images at predetermined time intervals and sequentially outputs captured images. Accordingly, the storage of the captured image in the memory 3 is sequentially updated.

  The control unit 2 includes a memory 3 and a CPU 4.

  The memory 3 is a storage unit that stores various data. The memory 3 stores, for example, information about the captured image output from the imaging unit 1 and the size of the small area set by the area dividing unit 21. The memory 3 further includes a noise model used for extraction of a target region for calculating a white balance gain (hereinafter referred to as “WB gain”), and a WB calculated by the WB gain calculation unit 25 and set in the WB adjustment unit 5. Also remember the gain.

  The CPU 4 is connected to the image sensor driving unit 12, the amplifying unit 14, the WB adjusting unit 5, and the image processing unit 6 via the data bus 9, and is also connected to the operation unit 7. The CPU 4 is a central processing unit that controls operations of these units. For example, the CPU 4 reads and executes a predetermined control program stored in advance in a read-only memory (not shown), thereby controlling operations of these units.

  In addition to this, the control unit 2 also includes a region dividing unit 21, a standard deviation calculating unit 22, an average value calculating unit 23, a flat region extracting unit 24, and a white balance gain calculating unit (hereinafter referred to as “WB gain calculating unit”). 25). Next, operations of these units will be described.

  In the imaging apparatus of FIG. 1, the captured image is divided into small areas, the standard deviation of the image signal of each pixel for each divided small area and for each color component is calculated, and the area on the captured image for calculating the WB gain is calculated. Then, extraction is performed based on the comparison result between the calculated standard deviation and the noise model.

  First, the noise model used for the above-described extraction will be described.

  An example of the noise model is shown in FIG. As illustrated in FIG. 2, the noise model includes a plurality of coordinate point data indicating the correspondence between the magnitude of the image signal and the amount of noise superimposed on the image signal. That is, the relationship between the exposure amount of the image sensor 11 and the noise expected to be superimposed on the image signal is quantified.

  The noise superimposed on the image signal is a variation in the image signal of each pixel generated due to the characteristics of the image sensor 11. The amount of noise that quantitatively represents this noise can be obtained by calculating the standard deviation of the image signal of each pixel. Note that noise generated by the characteristics of the image sensor 11 includes light shot noise, noise due to dark current, and readout noise. When the amount of incident light is large, light shot noise is dominant. Therefore, the amount of noise in this case depends on the amount of light incident on the image sensor 11.

  The amount of noise generated according to the magnitude of the image signal represented by this noise model can be obtained by actual measurement or by using a calculation formula for optical shot noise.

  In the calculation of the amount of noise by actual measurement, a partial range or the entire range of a uniform image only in the achromatic region is designated, and the image signal actually measured for each of the R, G, and B pixels included in the designated range is specified. A standard deviation is calculated, and this calculation result is used as a noise amount.

For example, the number of R, G, B pixels in the specified range is Nr, Ng, Nb, and the size of the image signal of each R, G, B pixel is Xri, Xgi, Xbi, and is included in the specified range. Let Xr _Ave , Xg _Ave , and Xb _Ave be the average values of the image signal sizes of the R, G, and B pixels. At this time, the noise amounts σr, σg, and σb with respect to the magnitudes of the image signals of the R, G, and B color components are respectively calculated by calculating the following [Equation 1].

  That is, the correspondence between the magnitude of the image signal and the amount of noise indicated by the noise model expressed by the equation 1 is included in a partial range or the entire range of a uniform image only in the achromatic region. This is a correspondence relationship between an average value and a standard deviation of image signals of pixels for each color component.

  On the other hand, when the amount of noise generated according to the magnitude of the image signal is obtained using a calculation formula for optical shot noise, the following is performed.

  The amount of light incident on the light receiving surface of the image sensor 11 is P, and the quantum efficiency of the image sensor 11 is Q. At this time, the amount of optical shot noise σS is calculated by calculating the following [Equation 2].

  For each of the R, G, and B color components, the optical shot noise is obtained by the above [Equation 2].

  For a plurality of images having only achromatic regions with different luminances, the amount of noise is obtained by the above [Equation 1] or [Equation 2], and the relationship between the magnitude of the image signal and the amount of noise is graphed. By doing so, a noise model is obtained. A data table corresponding to the noise model is stored in the memory 3 in advance.

  A noise model is prepared for each ISO sensitivity established by ISO (International Organization for Standardization), as illustrated in FIG. As the noise model to be compared with the standard deviation for each small region, the noise model having the ISO sensitivity set when the image is captured in the image capturing apparatus of FIG. 1 is used. The ISO sensitivity is a numerical value representing sensitivity to light incident on the imaging unit 1, and the control unit 2 sets a gain corresponding to the set ISO sensitivity in the amplification unit 14.

  Next, calculation of the standard deviation for each divided small region and each color component, which is compared with this noise model, will be described.

For example, the number of R, G, and B pixels in a small area obtained by dividing an image is NR, NG, and NB, and the image signal of each R, G, and B pixel is XRi, XGi, and XBi. XR _Ave , XG _Ave , and XB _Ave are average values of the magnitudes of the image signals of the R, G, and B pixels. At this time, the standard deviations σ R, σ G, and σ B of the image signal for each of the R, G, and B pixels in the small region are calculated by calculating the following [Equation 3], respectively.

  Returning to the description of FIG. The area dividing unit 21 divides a captured image into a plurality of small images based on the information on the image size specified by the operation on the operation unit 7 and stored in the memory 3 and the size of the small area stored in advance in the memory 3. Dividing into regions, coordinate data specifying each small region after the division is calculated. From the area dividing unit 21, the coordinate data and the horizontal and vertical sizes for each small area are output and input to the standard deviation calculating unit 22 and the average value calculating unit 23.

  The calculation of the coordinate data of the small area performed by the area dividing unit 21 will be described with reference to FIGS.

  FIG. 3 is an example of a captured image captured by the image capturing unit 1 and stored in the memory 3, and FIG. 4 illustrates a state in which the captured image of FIG. 3 is divided into small regions.

  The area dividing unit 21 first reads from the memory 3 the image size of the captured image of FIG. 3 designated by the operation on the operation unit 7 and the size of the small area stored in advance. Based on the read value, the coordinate data of each small area when this captured image is divided into small areas as shown in FIG. 4 is calculated.

  The image size represents the number of pixels in the horizontal direction and the vertical direction of the captured image, and is W (the number of pixels in the horizontal direction) and H (the number of pixels in the vertical direction) shown in FIG. .

  Further, the coordinate data of the small area is the horizontal and vertical coordinates of the upper left vertex of the rectangular small area, and the small area is specified by these coordinates. If this coordinate data is expressed in the form of (horizontal coordinate, vertical coordinate), for example, in FIG. 4, the coordinate data of the small region 501 is represented by (X0, Y0), and the small region 502 The coordinate data is represented by (X1, Y1).

  Also, the horizontal and vertical sizes of the small areas read out from the memory 3 and output by the area dividing unit 21 are, for example, w0 and h0 in FIG. Therefore, in FIG. 4, if the upper left vertex of the rectangular captured image is (0, 0), the coordinate data of the small region 501 is (0, 0), and the coordinate data of the small region 502 is (w0, h0).

  First, the standard deviation calculation unit 22 extracts image signals of pixels included in each small region from the captured image based on the coordinate data output from the region dividing unit 21. Then, the standard deviation of the pixel signal for each color component is calculated for each small area, and the calculation result is output in association with the coordinate data for specifying the small area.

  The average value calculation unit 23 extracts image signals of pixels included in each small region from the captured image based on the coordinate data output from the region dividing unit 21, and calculates the average value of the pixel signals for each color component as the small region. It calculates every time, and the calculation result is output in association with the coordinate data for specifying the small area.

  The flat area extracting unit 24 first compares the noise model stored in the memory 3 in advance with the standard deviation calculated by the standard deviation calculating unit 22 for each small area. Then, for the average value of the image signal for each color component of each small region calculated by the average value calculation unit 23, a small region in which the standard deviation is within a desired range based on the noise model is extracted. Coordinate data specifying the extracted small area is output as a flat area. In the present embodiment, this flat area is set as the WB gain calculation area.

  The extraction of a flat region (WB gain calculation region in the present embodiment) by the flat region extraction unit 24 will be further described.

  For example, consider the case in FIG. 2 where the ISO sensitivity set at the time of capturing an image in the image capturing apparatus is “ISO (1)”. At this time, the standard deviation σ with respect to the average value L for each color component of each small area in the noise model of “ISO (1)” in FIG. 2 is used as a reference, and up and down a% (1 ± a / 100). ) To extract a small area having a standard deviation within the range. The small area according to this extraction result is a small area in which the standard deviation is within a desired range with reference to the noise model with respect to the average value of the image signal for each color component of each small area, that is, a flat area.

  An example of flat region extraction by the flat region extraction unit 24 is shown in FIG.

  In FIG. 5, small areas 101 to 103 are background portions that are achromatic areas. The small areas 101 to 103 are flat areas whose standard deviation is within a desired range with respect to the average value of the image signal for each color component. Therefore, in the present embodiment, this flat region is extracted as a region for calculating the WB gain.

  On the other hand, in FIG. 5, although the small area 104 is also a background part, since the foreign substance exists in the area, the image signal includes the structure portion and noise on the structure portion. For this reason, the standard deviation of the image signal of the small area 104 is not within a desired range based on the noise model with respect to the average value of the image signal for each color component. Therefore, the small area 104 that is not a flat area is not extracted by the flat area extraction unit 24 as a WB gain calculation area.

  Returning to the description of FIG. The WB gain calculation unit 25 reads a captured image from the memory 3, identifies a small region for which a WB gain is calculated based on the coordinate data output by the flat region extraction unit 24, and pixels of the identified small region The WB gain is calculated using the image signal.

  Note that the CPU 4 can also be configured to provide the functions of the above-described region dividing unit 21, standard deviation calculating unit 22, average value calculating unit 23, flat region extracting unit 24, and WB gain calculating unit 25. For this purpose, for example, a control program for causing the CPU 4 to perform the function of each unit is created and stored in a read-only memory (not shown), and the CPU 4 reads out the control program when the imaging apparatus is activated. You should just run it.

  The operation unit 7 is connected to the control unit 2 and acquires an input of an instruction to start imaging by the imaging unit 1 that is associated with a predetermined operation. The acquired imaging start instruction is sent from the control unit 2 to the data bus 9 by the CPU 4.

  The WB adjustment unit 5 performs white balance adjustment (hereinafter referred to as “WB adjustment”) on the image signal output from the imaging unit 1 using the WB gain set by the control unit 2.

  The image processing unit 6 is connected to the WB adjustment unit 5 and the control unit 2 via the data bus 9. The image processing unit 6 performs various types of image processing such as color interpolation processing that compensates for missing color pixels in the color array, contour correction processing, and format conversion in the image signal that has been subjected to WB adjustment by the WB adjustment unit 5. Further, the image processing unit 6 generates an image signal that enables the display unit 8 to display an image and outputs the image signal to the display unit 8.

  The display unit 8 displays an image represented by this image signal.

  Next, a control process performed by the imaging apparatus in FIG. 1 will be described. FIG. 6 is a flowchart showing the processing content of the auto white balance processing according to the present embodiment performed by the control unit 2 of FIG.

  When the user of the imaging apparatus operates the operation unit 7 and inputs an instruction to start imaging, the CPU 4 acquires the input instruction to start imaging from the operation unit 7. Then, the CPU 4 performs processing for transferring this imaging start instruction to the imaging element driving unit 12 via the data bus 9. When acquiring the imaging start instruction, the imaging element driving unit 12 drives the imaging element 11 to start imaging. Then, data representing the captured image (captured image) is output from the imaging unit 1 and input to the control unit 2 and the WB adjustment unit 5. In the control unit 2, the CPU 4 stores the input captured image in the memory 3 and starts the control process of FIG.

  In FIG. 6, an area division process is performed in S1. In this process, the area dividing unit 21 is operated to read the image size and the size of the small area from the memory 3, and based on the read value, the captured image is divided into the small areas. This is a process for calculating coordinate data, which has already been described with reference to FIGS.

  Next, in S2, standard deviation calculation processing is performed. In this process, the standard deviation calculation unit 22 is made to function, and based on the coordinate data output from the region dividing unit 21, the image signal of the pixel in each small region is extracted from the captured image, and the standard deviation for each color component is calculated. This is a process of calculating for each small area. This calculation result is output from the standard deviation calculation unit 22 in association with coordinate data specifying the small area.

  Next, in S3, an average value calculation process is performed. In this process, the average value calculation unit 23 is made to function, and based on the coordinate data output from the region dividing unit 21, the image signals of the pixels in each small region are extracted from the captured image, and the average value for each color component is obtained. This is a process of calculating for each small area. This calculation result is output from the average value calculation unit 23 in association with coordinate data specifying the small area.

  Next, in S4, WB gain calculation region extraction processing is performed. This process is a process for causing the flat area extraction unit 24 to function. As described above, the flat region extraction unit 24 first compares the noise model stored in the memory 3 in advance with the standard deviation calculated by the standard deviation calculation unit 22 for each small region. Next, with respect to the average value of the image signal for each color component of each small region calculated by the average value calculation unit 23, a small region whose standard deviation is within a desired range based on the noise model is extracted. . Then, the coordinate data specifying the extracted small area is output as the extraction result of the WB gain calculation area.

  Next, in S5, a WB gain calculation process is performed. This process is a process for causing the WB gain calculation unit 25 to function. As described above, the WB gain calculation unit 25 first reads a captured image from the memory 3, and calculates a WB gain for the captured image based on the coordinate data output from the flat region extraction unit 24. Specify the small area to be used. Then, the WB gain is calculated using the image signal of the pixel in the specified small area. In calculating the WB gain, an average value of the image signals is obtained for each color component of the extracted WB gain calculation area, and the ratio of the average value of R to the average value of G and the ratio of the average value of B color are obtained. Calculate by taking. The WB gain calculation unit 25 temporarily stores the WB gain calculated in this way in the memory 3.

  In S6, WB adjustment processing is performed. In this process, the CPU 4 reads the WB gain stored by the WB gain calculation unit 25 from the memory 3 and sets it in the WB adjustment unit 5, and causes the WB adjustment unit 5 to perform WB adjustment. The WB adjustment unit 5 performs WB adjustment so that the average values of the R, G, and B color image signals in the WB gain calculation region are matched. When this WB adjustment process is completed, the auto white balance process ends.

  The above processing is the auto white balance processing of FIG. By causing the control unit 2 in FIG. 1 to perform this process, it is possible to provide an auto white balance function in the imaging apparatus in FIG.

  In the imaging apparatus according to the present embodiment, a single-plate color Bayer type is used as the imaging element 11, but a single-plate color imaging element other than the Bayer type may be used, and monochromatic imaging is also possible. You may comprise an imaging device using three elements.

  In the imaging apparatus according to the present embodiment, one noise model is used for each ISO sensitivity for the extraction of the WB gain calculation area. However, the noise model for each color component constituting the image signal for each ISO sensitivity is used. And a noise model corresponding to each color component may be used. In the imaging apparatus according to the present embodiment, the noise model is stored in the memory 3 as a data table. However, for example, an equation representing the noise model that is graphed is stored in the memory 3, and the flat region extraction unit 24 The flat area may be extracted using the noise amount obtained from the expression representing the noise model.

  In addition, the area dividing unit 21, the standard deviation calculating unit 22, the average value calculating unit 23, and the flat area extracting unit 24 included in the control unit 2 set the coordinate data and the size of the small region (the size in the horizontal direction and the vertical direction). In addition to outputting, image signals of pixels in each divided small area may be output.

  Further, the functions of the control unit 2, the WB adjustment unit 5, and the image processing unit 6 may be performed by, for example, a personal computer. In order to do this, a control program for causing the computer to perform the functions of these units may be created, and the control program may be read and executed by a personal computer.

  As described above, in this embodiment, the standard deviation for each color component of the image signal of the pixel in the small area obtained by dividing the image is compared with the noise model, and the WB gain calculation area is extracted based on the comparison result. In this embodiment, by doing in this way, it is possible to extract a structure part such as a subject or dust or an area where no noise such as overexposure or blackout exists as a WB gain calculation area. That is, since the background portion that is an achromatic region can be extracted with high accuracy as the WB gain calculation region, the WB gain can be calculated with high accuracy, and the accuracy of auto white balance adjustment is improved.

  [Second Embodiment]

  FIG. 7 is a configuration diagram of the imaging apparatus according to the present embodiment. In FIG. 7, the same reference numerals are assigned to components having the same functions as those in the first embodiment shown in FIG. Since these components have already been described, a detailed description thereof will be omitted here.

  The configuration of FIG. 7 is different from the configuration of FIG. 1 in that the control unit 2 includes a WB gain calculation optimum region extraction unit 26, a flat region re-extraction unit 27, and a warning unit 28.

  The WB gain calculation optimum region extraction unit 26 extracts a small region that matches at least one of the following two conditions as a WB gain calculation optimum region from each of the small regions extracted by the flat region extraction unit 24 as a flat region. To do. The first condition is a small region in which all of the average values of the image signals for each color component of each small region extracted by the flat region extraction unit 24 as a flat region are the maximum value. The second condition is that the average value of the image signal for each color component of each small area extracted by the flat area extraction unit 24 as a flat area is set based on the maximum value of the average value for each color component. This is a small area that is equal to or greater than the threshold value.

  The WB gain calculation optimum region extraction unit 26 extracts the WB gain calculation optimum region in this way because the flat region extracted by the flat region extraction unit 24 is limited to only the background portion that is an achromatic region. It is not. In other words, it is possible to extract the achromatic region with higher accuracy by further extracting the small region using the average value of the image signals for each color component of each small region extracted as a flat region. is there.

  Note that the threshold set with reference to the maximum value of the average value of the image signals for each color component of each small region is the average for each color component of the small region extracted as a flat region by the flat region extraction unit 24 here. Based on the maximum value among the values, the value is within a range of b% from the value.

  The extraction of the WB gain calculation optimum region performed by the flat region extraction unit 24 and the WB gain calculation optimum region extraction unit 26 will be further described with reference to FIG.

  FIG. 8 shows an example of an image obtained by imaging a specimen observed with a microscope. In FIG. 8, the divided small areas 201 to 203 are background parts which are achromatic areas, and the small areas 204 and 205 are areas where the thickness of the specimen is thin (areas which are not background parts).

  At this time, the flat area extraction unit 24 may extract all of the small areas 201 to 205 as a flat area. Here, the flat area extraction unit 24 extracts the small areas 204 and 205 when the specimen observed with a microscope is imaged, the area where the specimen is thin has a standard deviation comparable to that of the background portion. It is because there are things that are. However, since the small regions 204 and 205 are not background portions but regions where samples exist, it is not appropriate to use them for calculating the WB gain.

  On the other hand, the WB gain calculation optimum region extraction unit 26 selects a small region that matches at least one of the two conditions described above from each small region extracted by the flat region extraction unit 24 as a flat region. Extract as The extraction of the small area by the WB gain calculation optimum area extracting unit 26 will be further described with reference to FIGS. 9, 10, and 11. FIG.

  FIGS. 9, 10, and 11 are examples of graphs representing the average values of the image signals for the respective color components of the R color, the G color, and the B color in each small region.

  In FIG. 9, when the maximum value of the average value of the R color component image signal is Rmax, the threshold value set with reference to the maximum value Rmax is Rmax × (b / 100). In FIG. 10, if the maximum value of the average value of the G color component image signal is Gmax, the threshold value set based on the maximum value Gmax is Gmax × (b / 100). Further, in FIG. 11, when the maximum value of the average value of the B-color component image signal is Bmax, the threshold value set based on the maximum value Bmax is Bmax × (b / 100).

  Referring to FIGS. 9, 10, and 11, the small regions 201 to 203 satisfy at least one of the first and second conditions described above, and the small regions 204 and 205 satisfy the condition. Neither of these is met. Therefore, the WB gain calculation optimum area extraction unit 26 extracts the small areas 201 to 203. In this way, only the small areas 201 to 203 which are the background portions of the small areas 201 to 205 are extracted as the WB gain calculation optimum area.

  The flat area re-extraction unit 27 newly picks up an image under the predetermined condition when the flat area extraction in the flat area unit 24 fails (that is, when no flat area is extracted). The flat region is extracted again by the flat region unit 24 using the obtained image.

  The flat area re-extraction unit 27 first reads the number of times of flat area re-extraction stored in the memory 3 and counts up. Then, the counted number of times of flat area re-extraction is compared with the number of times of flat area re-extraction set previously stored in the memory 3.

  Here, if the counted number of times of flat area re-extraction is less than the number of times of setting flat area re-extraction, the flat area re-extraction unit 27 performs control for trying to extract the flat area again. That is, at this time, the flat region re-extraction unit 27 gives an instruction to re-calculate the coordinate data to the region dividing unit 21, and the re-calculation of the standard deviation calculation unit 22, the average value calculation unit 23, and the flat region extraction unit 24 thereafter. Start operation. In this way, the extraction of the flat region using the image newly picked up by the image pickup unit 1 is performed again.

  On the other hand, if the counted number of times of flat area re-extraction is equal to or greater than the set number of times of flat area re-extraction, it is determined that the auto white balance processing operation cannot be performed on the image, and the warning unit 28 is warned to the user. An instruction to notify is given. After that, the flat area re-extraction unit 27 gives an instruction to re-calculate the coordinate data to the area dividing unit 21, and the standard deviation calculation unit 22, the average value calculation unit 23, and the flat region extraction unit 24 are restarted thereafter. The flat area is extracted again from the image newly captured by the imaging unit 1.

  In response to an instruction from the flat area re-extraction unit 27, the warning unit 28 displays and outputs a warning indicating failure of auto white balance on the display unit 8 to notify the user.

  Next, a control process performed by the imaging apparatus in FIG. 7 will be described. FIG. 12 is a flowchart showing the processing content of the auto white balance processing according to this embodiment performed by the control unit 2 of FIG.

  In FIG. 12, the same reference numerals are assigned to the same processing steps as those according to the first embodiment shown in FIG. Since these processing steps have already been described, detailed description thereof is omitted here.

  The flowchart of FIG. 12 is different from the flowchart of FIG. 6 in that the processing steps from S7 to S11 and S18 are added, and the processing step of S4 is deleted instead.

  In FIG. 12, a flat area extraction process is performed in S7 following the average value calculation process in S3. In this determination process, the flat area extraction unit 24 is made to function, and first, the noise model stored in advance in the memory 3 and the standard deviation calculated by the standard deviation calculation unit 22 are compared for each small area. Then, with respect to the average value of the image signal for each color component of each small region calculated by the average value calculation unit 23, an attempt is made to extract a small region whose standard deviation is within a desired range based on the noise model. Then, it is determined whether or not the small area has been extracted. Here, when it is determined that the small region has been extracted (when the determination result is Yes), the coordinate data specifying the extracted small region is output to the flat region extraction unit 24 as the flat region extraction result. And the process proceeds to S18. This flat region extraction result is input to the WB gain calculation optimum region extraction unit 26. On the other hand, when it is determined that the small area cannot be extracted (when the determination result is No), the flat area re-extraction signal is output to the flat area extracting unit 24, and the process proceeds to S9. The flat area re-extraction signal is input to the flat area re-extraction unit 27.

  In S18, flat area re-extraction count reset processing is performed. This process is a process of resetting the flat area re-extraction count stored in the memory 3 to the initial value “0” because the small area that is a flat area has been extracted. In subsequent S8, WB gain calculation optimum region extraction processing is performed.

  This WB gain calculation optimum region extraction process is a process for causing the WB gain calculation optimum region extraction unit 26 to function. As described above, the WB gain calculation optimum region extraction unit 26 selects from the small regions extracted by the flat region extraction unit 24 as flat regions, the small regions that meet at least one of the first and second conditions described above. Are extracted as the WB gain calculation optimum region. The WB gain calculation optimum region extraction unit 26 outputs the coordinate data specifying the small region thus extracted as the extraction result of the WB gain calculation optimum region.

  In the WB gain calculation process of S5 performed following the process of S8, the WB gain is calculated for the captured image read from the memory 3 based on the coordinate data output from the WB gain calculation optimum region extraction unit 26. Specify the target small area. Then, the WB gain is calculated using the image signal of the pixel in the specified small area, and the calculated WB gain is temporarily stored in the memory 3. Then, in the subsequent WB adjustment process of S6, the CPU 4 reads the WB gain stored by the WB gain calculation unit 25 from the memory 3 and sets it in the WB adjustment unit 5, and causes the WB adjustment unit 5 to perform WB adjustment.

  On the other hand, in S9, flat area re-extraction count counting processing is performed. As described above, this processing is performed when the flat region portion 24 cannot extract a small region and it is determined in S7 that the flat region extraction has failed. In this process, the flat area re-extraction unit 27 is caused to function so that the number of times of flat area re-extraction stored in the memory 3 is read and incremented by “1”. 3 and the value is updated.

  Next, in S10, flat area re-extraction number determination processing is performed. In this process, the flat area re-extraction unit 27 is caused to function so as to read the flat area re-extraction setting number stored in the memory 3 in advance, and the flat area re-extraction after counting up by the setting number and the process of S9. Make a comparison with the number of times. Here, when it is determined that the number of times of flat area re-extraction after the count-up is less than the number of flat area re-extraction settings (when the determination result is No), the process returns to S1 to extract the flat area. The processes from S1 to S3 and S7 are started again. On the other hand, when it is determined that the number of flat region re-extraction after the count-up is equal to or greater than the flat region re-extraction setting number (when the determination result is Yes), the process proceeds to S11.

  In S11, an auto white balance failure warning process is performed. This process is a process for causing the warning unit 28 to function, and is a process for causing the display unit 8 to display and output a warning indicating failure of auto white balance and notifying the user of the warning. Thereafter, when the process of S11 is completed, the process is returned to S1, and the processes of S1 to S3 and S7 for extracting the flat area are started again.

  The above processing is the auto white balance processing of FIG. By causing the control unit 2 in FIG. 7 to perform this process, it is possible to provide an auto white balance function in the imaging apparatus in FIG.

  In the above description of the present embodiment, it is assumed that a background portion that is an achromatic region always exists in an image, but there may be a case where such a background portion does not exist. Therefore, the imaging apparatus of FIG. 7 may be operated as follows.

  That is, first, the maximum value of the average value of the image signals for each color component of each small area extracted as the WB gain calculation optimum area when auto white balance processing is performed for the first time at the start of imaging is stored in the memory 3. . After that, every time auto white balance processing is executed, the maximum value of the average value of the image signal for each color component of each small area obtained by the execution is compared with the stored maximum value, and this value is compared. The stored contents of the memory 3 are updated with the larger of the two as the maximum value. When extracting a small region having an average value equal to or greater than a threshold value set based on the maximum value of the average value of the image signal for each color component at the time of subsequent extraction of the WB gain calculation optimum region, The maximum value stored in the memory 3 is used as a reference for setting the threshold.

  Even when the image pickup apparatus of FIG. 7 is operated as described above, the background portion that is the achromatic region can be extracted with high accuracy.

  Further, in the process of S11 in the auto white balance process shown in FIG. 12, when it is determined that the auto white balance process has failed (when the determination result is Yes), the WB adjustment process is finally performed. You may make it perform WB adjustment using WB gain. At this time, if it is determined that the process has failed in the first auto white balance process after imaging is started, the WB adjustment is performed using the initial setting value of the WB gain stored in the memory 3 in advance. May be performed.

  As described above, in the present embodiment, the extraction of the WB gain calculation optimum region is not only performed based on the comparison result between the standard deviation of the image signal for each color component of each small region and the noise model, but also the standard deviation. Is also performed based on the average value of the image signals for each color component of each small region extracted using. In this embodiment, since the background portion that is the achromatic region can be extracted with higher accuracy by doing so, the WB gain can be calculated with higher accuracy. The accuracy of white balance adjustment is further improved.

  Further, in the present embodiment, when the flat area extraction fails, the user's work load can be reduced by trying to re-extract the flat area. Here, when the re-extraction of the flat area has failed for the preset number of times, the user is warned that the auto white balance processing has failed, so the user recognizes that the WB adjustment is not performed. Can be made. As a result, it is possible to reduce imaging failures due to mistakes in WB adjustment.

  [Third embodiment]

  FIG. 13 is a configuration diagram of the imaging apparatus according to the present embodiment. In FIG. 13, components having the same functions as those according to the first embodiment shown in FIG. 1 or those according to the second embodiment shown in FIG. doing. Since these components have already been described, a detailed description thereof will be omitted here.

  The configuration of FIG. 13 is different from the configuration of FIG. 7 in that the control unit 2 includes a WB gain calculation optimum region re-extraction unit 29.

  When it is determined that the extraction of the WB gain calculation optimal region has failed, the WB gain calculation optimal region re-extraction unit 29 calculates the WB gain using the image newly captured by the imaging unit 1 under a predetermined condition. The optimum area extraction unit 26 is made to extract the WB gain calculation optimum area again.

  The WB gain calculation optimum region re-extraction unit 29 first reads out the WB gain calculation optimum region re-extraction number stored in the memory 3 and counts up. Then, the counted WB gain calculation optimum region re-extraction number is compared with the WB gain calculation optimum region re-extraction setting number stored in the memory 3 in advance.

  Here, if the counted WB gain calculation optimum region re-extraction number is less than the WB gain calculation optimum region re-extraction setting number, the WB gain calculation optimum region re-extraction unit 29 again extracts the WB gain calculation optimum region. Take control to try. That is, at this time, the WB gain calculation optimum region re-extraction unit 29 gives an instruction to re-calculate coordinate data to the region division unit 21. As a result, the subsequent re-operations of the standard deviation calculation unit 22, the average value calculation unit 23, the flat region extraction unit 24, and the WB gain calculation optimum region extraction unit 26 are started. In this way, the extraction of the WB gain calculation optimum region using the image newly picked up by the image pickup unit 1 is performed again.

  On the other hand, if the counted WB gain calculation optimum region re-extraction number is equal to or more than the WB gain calculation optimum region re-extraction setting number, it is determined that the autobalance processing operation for this image is impossible, and the warning unit 28 is notified. An instruction to notify the user of a warning is given. In addition, the WB gain calculation optimum region re-extraction unit 29 gives an instruction to re-calculate coordinate data to the region division unit 21. As a result, the standard deviation calculation unit 22, the average value calculation unit 23, the flat region extraction unit 24, and the WB gain calculation optimum region extraction unit 26 are restarted, and the image newly captured by the imaging unit 1 is used. The extracted WB gain calculation optimum region is again performed.

  Next, control processing according to the present embodiment performed by the imaging apparatus in FIG. 13 will be described. FIG. 14 is a flowchart showing the processing content of the auto white balance processing according to this embodiment performed by the control unit 2 of FIG.

  14, the same reference numerals are assigned to the same processing steps as those according to the first embodiment shown in FIG. 6 or according to the second embodiment shown in FIG. Since these processing steps have already been described, detailed description thereof is omitted here.

  The flowchart of FIG. 14 is different from the flowchart of FIG. 12 in that the processing steps from S12 to S14 and S19 are added and the processing step of S8 is deleted instead.

  In FIG. 14, a WB gain calculation optimum region extraction process is performed as a process of S12 following the flat region re-extraction count reset process of S18. This process is a process for causing the WB gain calculation optimum region extraction unit 26 to function.

  The WB gain calculation optimum region extraction unit 26 firstly performs at least one of the first and second conditions described above from each small region extracted as a flat region by the flat region extraction unit 24 in the same manner as S8 in FIG. Attempt to extract a small region that matches one. Then, it is determined whether or not this small area has been extracted. Here, when it is determined that the small area has been extracted (when the determination result is Yes), the coordinate data specifying the extracted small area is output as the extraction result of the WB gain calculation optimum area, and the process is performed. Proceed to S19. The extraction result of the WB gain calculation optimum region is input to the WB gain calculation optimum region extraction unit 26. On the other hand, when it is determined that the small area cannot be extracted (when the determination result is No), the WB gain calculation optimum area extraction unit 26 outputs the WB gain calculation optimum area re-extraction signal, and the process proceeds to S9. . The WB gain calculation optimum region re-extraction signal is input to the WB gain calculation optimum region re-extraction unit 29.

  In S19, a WB gain calculation optimum region re-extraction number reset process is performed. This process is a process for resetting the WB gain calculation optimum region re-extraction count stored in the memory 3 to the initial value “0” because the small region that is the WB gain calculation optimum region has been extracted. .

  In the WB gain calculation process of S5 performed following the process of S19, the WB gain is calculated for the captured image read from the memory 3 based on the coordinate data output from the WB gain calculation optimum region extraction unit 26. Specify the target small area. Then, the WB gain is calculated using the image signal of the pixel in the specified small area, and the calculated WB gain is temporarily stored in the memory 3. Then, in the subsequent WB adjustment process of S6, the CPU 4 reads the WB gain stored by the WB gain calculation unit 25 from the memory 3 and sets it in the WB adjustment unit 5, and causes the WB adjustment unit 5 to perform WB adjustment.

  On the other hand, in S13, the WB gain calculation optimum area re-extraction count processing is performed. As described above, this processing is performed when the WB gain calculation optimum region extraction unit 26 cannot extract a small region and it is determined in step S7 that the extraction of the WB gain calculation optimum region has failed. . In this process, the WB gain calculation optimum region re-extraction unit 29 is caused to function, and first, the number of WB gain calculation optimum region re-extraction stored in the memory 3 is read and incremented by “1”. Then, the WB gain calculation optimum region re-extraction count after the count-up is stored in the memory 3 and the value is updated.

  Next, in S14, a WB gain calculation optimum region re-extraction number determination process is performed. In this processing, the WB gain calculation optimum region re-extraction unit 29 is caused to function, and first, the number of WB gain calculation optimum region re-extraction settings stored in advance in the memory 3 is read. Then, the set number of times is compared with the number of times of WB gain calculation optimum region re-extraction after the count-up by the process of S13. Here, when it is determined that the WB gain calculation optimum region re-extraction number after the count-up is less than the WB gain calculation optimum region re-extraction setting number (when the determination result is No), the process returns to S1. Then, the processing from S1 to S3, S7, S18, and S12 for extracting the WB gain calculation optimum region is started again. On the other hand, when it is determined that the WB gain calculation optimum region re-extraction count after the count-up is equal to or greater than the WB gain calculation optimum region re-extraction setting count (when the determination result is Yes), the process proceeds to S11.

  The auto white balance failure warning process in S11 is the same as that shown in FIG. 12, and the warning unit 28 is caused to function to display a warning indicating the auto white balance failure on the display unit 8 to notify the user of the warning. It is processing to do. Thereafter, when the process of S11 is completed, the process is returned to S1, and the processes of S1 to S3, S7, S18, and S12 for extracting the WB gain calculation optimum region are started again.

  The above processing is the auto white balance processing of FIG. By causing the control unit 2 in FIG. 13 to perform this process, it is possible to provide an auto white balance function in the imaging apparatus in FIG.

  As described above, in the present embodiment, when the WB gain calculation optimum region cannot be extracted, by trying to re-extract the WB gain calculation optimum region, a portion other than the achromatic region can be accurately extracted. Thus, the extraction of the WB gain calculation optimum region can be performed with higher accuracy. Further, in this embodiment, when there is no achromatic region (that is, the WB gain calculation optimum region) in the small region extracted as the flat region, the user is warned that the auto white balance process has failed, so the WB adjustment is correctly performed. The user can accurately recognize that this is not done. As a result, it is possible to further reduce imaging failure due to a mistake in WB adjustment.

  [Fourth embodiment]

  In the embodiments described so far, the size of the small area when the area dividing unit 21 divides the area of the image is set as a fixed value. However, the shape of the subject to be imaged and the size of the background portion are various. Therefore, in the present embodiment, the size of the small area when the area dividing unit 21 divides the image area is variable, and the size can be set by the user.

  The configuration of the imaging apparatus according to the present embodiment is the same as that according to the third embodiment shown in FIG. Therefore, since each component in the imaging apparatus according to the present embodiment has been described, detailed description thereof is omitted here.

  A control process according to this embodiment performed by the imaging apparatus of FIG. 13 will be described. FIG. 15 is a flowchart showing the processing content of the auto white balance processing according to the present embodiment performed by the control unit 2 of FIG.

  In FIG. 15, the same processing steps as those included in the flowchart shown in FIG. 6, FIG. 12, or FIG. Since these processing steps have already been described, detailed description thereof is omitted here.

  The flowchart of FIG. 15 is different from the flowchart of FIG. 14 in that processing steps S15 and S16 are added at the beginning of the process.

  Before the processing of FIG. 15 is started, an initial value of the size of the small area when the area dividing unit 21 divides the image area is stored in the memory 3 in advance.

  When the process of FIG. 15 is started, first, in S15, a process of setting the size of the divided area is performed.

  In this process, first, the CPU 4 acquires an input of an instruction for the size in the horizontal direction and the vertical direction of the small area, which is performed by the user operating the operation unit 7. Then, the CPU 4 transfers the data of the size according to this instruction to the memory 3 and stores it, and updates the data related to the size of the small area stored therein.

  In the area dividing process of S1 that is first executed after the process of S15, the area dividing unit 21 is caused to function so that the updated data is read from the memory 3, and based on this data, the captured image is converted into a small area. A process of calculating the coordinate data of each small area when divided into two is performed. The division of the captured image into small regions will be described with reference to FIG.

  FIG. 16 is a second example in which the captured image illustrated in FIG. 3 is divided into small regions. FIG. 16 shows the case where the user sets the horizontal size X and the vertical size Y of the small area to X = w1 and Y = h1, respectively, and the captured image is in the horizontal direction and the vertical direction. The direction is divided into small regions of X = w1 and Y = h1, respectively.

  Subsequent to the process of S15, in S16, a process of selecting a range on the captured image for obtaining a small area to be calculated by the standard deviation calculation process of S2 and the average value calculation process of S3 is performed.

  In this process, first, the CPU 4 acquires an instruction for a range on the captured image, which is performed by the user operating the operation unit 7. Then, the CPU 4 transfers the data in the range according to this instruction to the memory 3 for storage.

  In the area dividing process of S1 that is executed first after the process of S16, the area dividing unit 21 is caused to function, and first, this data is read from the memory 3. Then, based on this data, an image in the range according to this instruction is selected from the captured image, and a process of calculating coordinate data of each small region when the image in the selected range is divided into small regions is performed. The selection of the standard deviation and average value calculation target range will be described with reference to FIG.

  FIG. 17 illustrates a state where the calculation range of the standard deviation and the average value is selected from the captured image illustrated in FIG. FIG. 17 shows a case where the user designates a small area range 301 which is a calculation target range of standard deviation and average value. In this case, the area dividing unit 21 outputs the coordinate data of each small area existing in the small area range 301.

  After completion of the processes of S15 and S16, the same process as the process of FIG. 14 described above is performed. By causing the control unit 2 in FIG. 14 to perform the auto white balance process in FIG. 15, it is possible to provide an auto white balance function in the imaging apparatus in FIG.

  As described above, in the present embodiment, for example, in the case of a captured image in which the proportion of the subject is large and the background portion that is an achromatic region is small, it is possible to efficiently reduce the small background portion by changing the size of the small region. Can be extracted.

  Also, when the number of small areas increases because the size of the small areas is small, it takes a long time to calculate the standard deviation and average value of each small area, to extract the flat area and to calculate the optimal area of the WB gain. It becomes. In such a case, according to the present embodiment, these processing times can be shortened by selecting in advance a small region that is a target for calculating the standard deviation and the average value.

  In the present embodiment, the shape of the small area to be divided is a rectangle as shown in FIG. 17, but this shape may be any shape such as a circle or an ellipse. Also, the captured image may be divided so that a desired region is obtained instead of the lattice shape. Further, the divided image is displayed on the display unit 8, and the user operates the operation unit 7 while referring to the display on the display unit 8. When the size is designated, the image may be divided by the designated size. Alternatively, the user may select a region for calculating the standard deviation and the average value by operating the operation unit 7 while viewing the region-divided image displayed on the display unit 8.

  [Fifth embodiment]

  Assume that a microscopic image of an observation specimen observed using a microscope is captured. FIG. 18 is an example of an image obtained by imaging the specimen shown in the image illustrated in FIG. 3 using a high-magnification objective lens.

  As can be seen by comparing the captured image of FIG. 18 with the image of FIG. 3, when a subject is imaged using a high-magnification objective lens, the proportion of the subject in the captured image depends on the magnification of the objective lens. Change.

  Therefore, in the present embodiment, the horizontal and vertical sizes of the small areas to be divided are changed according to the magnification of the objective lens used for observing the subject.

  FIG. 19 is a configuration diagram of a microscope system including the imaging apparatus according to the present embodiment. In FIG. 19, components having the same functions as those in the imaging apparatus shown in FIG. 1, FIG. 7, or FIG. Since these components have already been described, a detailed description thereof will be omitted here.

  The configuration of FIG. 19 is different from the configuration of FIG. 13 in that the imaging unit 1 is installed in a microscope 33 equipped with the objective lens 32 and an observation condition acquisition unit 34 is added. Yes. An observation specimen 31 is placed on the microscope 33, and a microscope image of the observation specimen 31 is obtained. Further, the imaging element 11, the imaging element driving unit 12, the pre-processing unit 13, the amplification unit 14, and the A / D conversion unit 15 which are components of the imaging unit 1 are not illustrated in FIG.

  In FIG. 19, an observation image of the observation specimen 31 is picked up by the image pickup unit 1 through the objective lens 32. The objective lens 32 is used by being switched by the user when observing the observation specimen 31.

  The observation condition acquisition unit 34 acquires the setting of the microscope 33 during observation of the observation specimen 31 as an observation condition via a signal line connecting the microscope 33 and the observation condition acquisition unit 34. The observation condition acquisition unit 34 sends the acquired observation conditions to the data bus 9 in accordance with an instruction from the CPU 4 in the control unit 2. The sent observation conditions are transferred to the control unit 2.

  In the present embodiment, the observation condition acquisition unit 34 obtains objective lens magnification information including information for specifying the objective lens 32 used for observation of the observation specimen 31 and information on the magnification (that is, observation magnification). It shall be acquired as an observation condition. This objective lens magnification information is used for setting the size of the small area when the area dividing unit 21 divides the captured image into small areas.

  Next, control processing according to the present embodiment performed by the imaging apparatus in FIG. 19 will be described. FIG. 20 is a flowchart showing the processing content of the auto white balance processing according to this embodiment performed by the control unit 2 of FIG.

  20, the same processing steps as those included in the flowchart shown in FIG. 6, FIG. 12, FIG. 14 or FIG. Since these processing steps have already been described, detailed description thereof is omitted here.

  The flowchart of FIG. 20 is different from the flowchart of FIG. 15 in that the processing step of S17 is added at the beginning of the process.

  When the processing of FIG. 17 is started, first, microscope information acquisition processing is performed in S17. In this process, first, the observation condition acquisition unit 34 is caused to function so that the above-described objective lens magnification information is acquired as an observation condition and transferred to the control unit 2. The control unit 2 acquires the objective lens magnification information, calculates the horizontal and vertical sizes of the small area when dividing the captured image based on the objective lens magnification information, and stores them in the memory 3. Is performed by the CPU 4. The calculation of the size of the small area in the horizontal direction and the vertical direction is performed by the CPU 4 as follows, for example.

  First, the size in the horizontal direction and the vertical direction of the optimal small region when using the objective lens 32 equipped in the microscope 33 with the lowest magnification is stored in the memory 3 in advance. If the objective lens magnification information indicates that an objective lens 32 other than the minimum magnification is used, the ratio of the magnification to the minimum magnification is obtained. Then, the horizontal and vertical sizes of the small areas stored in the memory 3 are changed according to this ratio.

  For example, when the objective lens 32 having the minimum magnification of M, which is stored in the memory 3 in advance, is used, the optimum size of the small region in the horizontal direction and the vertical direction is x and y, respectively. At this time, when the objective lens magnification information acquired by the observation condition acquisition unit 34 indicates the use of the objective lens 32 having a magnification of N times, the sizes of the small area in the horizontal direction and the vertical direction are expressed as follows: Change to k × (M / N) × x and k × (M / N) × y, respectively. Here, k is a predetermined constant.

  After completion of the process of S17, the same process as the process of FIG. 15 described above is performed. By causing the control unit 2 in FIG. 19 to perform the autobalance process in FIG. 20, it is possible to provide an autobalance function in the imaging apparatus in FIG.

  FIG. 21 shows a state in which the captured image of FIG. 18 is divided into small regions by the microscope system according to the present embodiment. When the image of FIG. 21 is compared with the image of FIG. 4, the image of FIG. 21 uses the objective lens 32 having a higher magnification than that used when the image of FIG. It can also be seen that the size in the vertical direction is smaller than the image in FIG.

  In the captured image when the magnification of the objective lens 32 is increased, the observation sample 31 occupies most of the image, and the background portion that is an achromatic region is reduced. Therefore, extraction of the WB gain calculation optimum region is likely to fail. End up. Therefore, in the present embodiment, in such a case, the possibility of extracting the optimum WB gain calculation region is increased by reducing the size of the small region in the horizontal direction and the vertical direction. In the image example of FIG. 21, it can be seen that the small regions 401 to 407 can be extracted as the WB gain calculation optimum region.

  As described above, in the present embodiment, when the objective lens 32 to be used is changed, the size of the small region in the horizontal direction and the vertical direction is changed according to the magnification of the objective lens 32. According to this embodiment, by doing in this way, the background part which is an achromatic area | region where the ratio occupied to a captured image decreases can be extracted efficiently.

  In the present embodiment, the shape of the small area to be divided is a rectangle as shown in FIG. 21, but this shape may be any shape such as a circle or an ellipse.

  In the present embodiment, the size of the small area is calculated based on the ratio of magnifications of the objective lens 32 used for capturing an image. Instead of this, a table in which the optimum size of the small area corresponding to the magnification is set in advance is prepared, and the size of the small area is determined based on the obtained observation conditions with reference to this table. It may be.

  In addition, instead of selecting the range of the small area, which is the calculation target range of the standard deviation and the average value, based on the user's instruction as shown in FIG. It is also possible to configure the imaging device to automatically select based on (1).

DESCRIPTION OF SYMBOLS 1 Imaging part 2 Control part 3 Memory 4 CPU
5 WB adjustment unit 6 Image processing unit 7 Operation unit 8 Display unit 9 Data bus 11 Image sensor 12 Image sensor drive unit 13 Pre-processing unit 14 Amplification unit 15 A / D conversion unit 21 Area division unit 22 Standard deviation calculation unit 23 Average Value calculation unit 24 Flat region extraction unit 25 WB gain calculation unit 26 WB gain calculation optimum region extraction unit 27 Flat region re-extraction unit 28 Warning unit 29 WB gain calculation optimum region re-extraction unit 31 Observation specimen 32 Objective lens 33 Microscope 34 Observation condition Acquisition department

Claims (7)

An image pickup unit that includes an image pickup device that picks up an image of a subject, and that generates a picked-up image of the subject in which colors are expressed for each pixel by image signals of a plurality of color components;
Area dividing means for dividing the captured image into a plurality of small areas;
A standard deviation calculating means for calculating a standard deviation of an image signal of a pixel for each color component for each pixel included in each of the plurality of small regions in the captured image;
An average value calculating means for calculating an average value of image signals of pixels for each color component for each pixel included in each of the plurality of small regions in the captured image;
Based on a noise model that is stored in advance in a storage unit and indicates a correspondence relationship between the magnitude of the image signal and the amount of noise generated by the imaging element superimposed on the image signal, the plurality of images in the captured image Among the small areas, the small area whose standard deviation with respect to the average value is within a predetermined range based on the amount of noise when the average value is regarded as the size of the image signal in the noise model is extracted as a flat area. Flat area extracting means for
White balance gain calculating means for calculating a white balance gain of the flat area in the captured image;
White balance adjustment means for adjusting white balance of a captured image generated by the imaging means based on the white balance gain of the flat region calculated by the white balance gain calculation means;
Among the small areas extracted as flat areas by the flat area extracting means, all the average values of the image signals for each color component are the maximum values, or the average value is the average value for each color component. White balance gain calculation optimum region extraction means for extracting a small region that is equal to or larger than a threshold set with reference to the maximum value of the white balance gain calculation optimum region;
I have a,
The white balance gain calculation means calculates a white balance gain of the white balance gain calculation optimum region in the captured image,
The white balance adjustment unit adjusts the white balance of the captured image generated by the imaging unit based on the white balance gain of the white balance gain calculation optimum region calculated by the white balance gain calculation unit.
An imaging apparatus characterized by that. An image pickup unit that includes an image pickup device that picks up an image of a subject, and that generates a picked-up image of the subject in which colors are expressed for each pixel by image signals of a plurality of color components;
Area dividing means for dividing the captured image into a plurality of small areas;
A standard deviation calculating means for calculating a standard deviation of an image signal of a pixel for each color component for each pixel included in each of the plurality of small regions in the captured image;
An average value calculating means for calculating an average value of image signals of pixels for each color component for each pixel included in each of the plurality of small regions in the captured image;
Based on a noise model that is stored in advance in a storage unit and indicates a correspondence relationship between the magnitude of the image signal and the amount of noise generated by the imaging element superimposed on the image signal, the plurality of images in the captured image Among the small areas, the small area whose standard deviation with respect to the average value is within a predetermined range based on the amount of noise when the average value is regarded as the size of the image signal in the noise model is extracted as a flat area. Flat area extracting means for
White balance gain calculating means for calculating a white balance gain of the flat area in the captured image;
White balance adjustment means for adjusting white balance of a captured image generated by the imaging means based on the white balance gain of the flat region calculated by the white balance gain calculation means;
Flat area extraction determining means for determining whether or not the flat area extracting means has extracted a small area that is the flat area;
When the flat region extraction determining unit determines that the flat region extracting unit cannot extract the small region that is the flat region, the flat region extracting unit re-extracts the small region that is the flat region by the flat region extracting unit. Flat area re-extraction means for performing another captured image of the subject generated by the imaging means;
An imaging device comprising: A flat region re-extraction number determination unit that determines whether or not the number of times of re-extraction of the small region that is the flat region that the flat region re-extraction unit has performed is the predetermined number or more;
Warning means for outputting a warning of auto white balance failure when the flat area re-extraction number determination means determines that the number of times of re-extraction of the small area that is the flat area is equal to or greater than a predetermined number;
The imaging apparatus according to claim 2 , further comprising: A white balance gain calculation optimum region extraction determination unit that determines whether or not the white balance gain calculation optimum region extraction unit has extracted a small region that is the white balance gain calculation optimum region;
When the white balance gain calculation optimum region extraction means determines that the white balance gain calculation optimum region extraction means cannot extract the small region that is the white balance gain calculation optimum region, the white balance gain calculation optimum region A white balance gain calculation optimum region re-extraction unit that causes re-extraction of the small region, which is the white balance gain calculation optimum region by the extraction unit, to another captured image of the subject generated by the imaging unit;
The imaging apparatus according to claim 1 , further comprising: Whether or not the number of times of re-extraction of the small region, which is the white balance gain calculation optimum region, which the white balance gain calculation optimum region re-extraction unit has performed by the white balance gain calculation optimum region re-extraction unit has reached a predetermined number or more. White balance gain calculation optimal region re-extraction number determination means for determining;
When the white balance gain calculation optimum region re-extraction number determination means determines that the number of times of re-extraction of the small region, which is the white balance gain calculation optimum region, exceeds a predetermined number, a warning indicating auto white balance failure is output Warning means to
The imaging apparatus according to claim 4 , further comprising: An image pickup unit that includes an image pickup device that picks up an image of a subject, and that generates a picked-up image of the subject in which colors are expressed for each pixel by image signals of a plurality of color components;
Area dividing means for dividing the captured image into a plurality of small areas;
A standard deviation calculating means for calculating a standard deviation of an image signal of a pixel for each color component for each pixel included in each of the plurality of small regions in the captured image;
An average value calculating means for calculating an average value of image signals of pixels for each color component for each pixel included in each of the plurality of small regions in the captured image;
Based on a noise model that is stored in advance in a storage unit and indicates a correspondence relationship between the magnitude of the image signal and the amount of noise generated by the imaging element superimposed on the image signal, the plurality of images in the captured image Among the small areas, the small area whose standard deviation with respect to the average value is within a predetermined range based on the amount of noise when the average value is regarded as the size of the image signal in the noise model is extracted as a flat area. Flat area extracting means for
White balance gain calculating means for calculating a white balance gain of the flat area in the captured image;
White balance adjustment means for adjusting white balance of a captured image generated by the imaging means based on the white balance gain of the flat region calculated by the white balance gain calculation means;
I have a,
The imaging means captures a microscope image of an observation specimen obtained by a microscope with the imaging element, and generates a captured image of the observation specimen,
It further has observation condition acquisition means for acquiring information on observation conditions of the microscope when imaging a microscope image of the observation specimen,
The information on the observation condition includes information on the observation magnification of the microscope,
The area dividing means divides the captured image by the size of the small area changed according to the information of the observation magnification included in the observation condition acquired by the observation condition acquiring means.
An imaging apparatus characterized by that. A microscope for obtaining a microscopic image of the observation specimen;
An image pickup unit that includes an image pickup device that picks up the microscope image obtained by the microscope, and that generates a picked-up image of the observation specimen in which colors are expressed for each pixel by image signals of a plurality of color components;
Area dividing means for dividing the captured image into a plurality of small areas;
A standard deviation calculating means for calculating a standard deviation of an image signal of a pixel for each color component for each pixel included in each of the plurality of small regions in the captured image;
An average value calculating means for calculating an average value of image signals of pixels for each color component for each pixel included in each of the plurality of small regions in the captured image;
Based on a noise model that is stored in advance in a storage unit and indicates a correspondence relationship between the magnitude of the image signal and the amount of noise generated by the imaging element superimposed on the image signal, the plurality of images in the captured image Among the small areas, the small area whose standard deviation with respect to the average value is within a predetermined range based on the amount of noise when the average value is regarded as the size of the image signal in the noise model is extracted as a flat area. Flat area extracting means for
White balance gain calculating means for calculating a white balance gain of the flat area in the captured image;
White balance adjustment means for adjusting white balance of a captured image generated by the imaging means based on the white balance gain of the flat region calculated by the white balance gain calculation means;
Observation condition acquisition means for acquiring information on observation conditions of the microscope when imaging a microscope image of the observation specimen;
Have
The information on the observation condition includes information on the observation magnification of the microscope,
The area dividing means divides the captured image by the size of the small area changed according to the information of the observation magnification included in the observation condition acquired by the observation condition acquiring means.
A microscope system characterized by that.