WO2023210619A1 - Imaging device, inspection device, imaging condition determination method, and imaging method - Google Patents

Abstract

RGB pixels of a camera (31) include a first pixel having sensitivity to a first wavelength region and a second pixel having sensitivity to a second wavelength region. The first wavelength region and the second wavelength region overlap in a partial overlapping wavelength region. Monochromatic light is light in an emission wavelength region including at least a part of the first wavelength region and at least a part of the overlapping wavelength region. An image processing unit (50) comprises an arithmetic processing unit (52) and an amplification unit (53). The arithmetic processing unit (52) performs linear matrix arithmetic processing on a three-band RGB image from the camera (31) to generate a three-band XYZ image having corrected RGB imaging characteristics. The gain of the amplification unit (53) is set to a value that enables the output levels of the first pixel and the second pixel to be color-balanced between two colors under monochromatic light illumination.

Description

Imaging device, inspection device, imaging condition determination method, and imaging method

The present invention relates to an imaging device, an inspection device, an imaging condition determination method, and an imaging method that output images of different types of similar-colored objects in images of objects as images that are easy to identify.

For example, Patent Documents 1 to 3 disclose inspection devices that include an imaging device that images, for example, a colony as an imaging target. In Patent Document 1, the number of colonies of microorganisms cultured in a petri dish can be accurately counted, down to minute colonies of various shapes, using a simple device configuration using a black-and-white CCD camera, without requiring complicated pretreatment of the culture medium. Count in a short time.

Patent Documents 2 and 3 disclose colony detection systems that can accurately identify two or more types of colonies that have different color features. This colony detection system classifies colony pixels into predetermined types based on the color features of colony pixels in order to accurately identify two or more types of colonies with different color features. identify.

By this, it is possible to distinguish between two or more types of colonies with different color characteristics, such as reddish-purple or navy blue, which cannot be distinguished by simply detecting colonies using gray scale, or to determine the bacteria to which an intermediate color belongs. When identifying target bacteria and bacteria similar to the target bacteria, detection is possible even when the color characteristics are similar.

Japanese Patent Application Publication No. 2006-345750 Japanese Patent Application Publication No. 2017-035042 Unexamined Japanese Patent Publication No. 2016-26500

However, there may be cases where there are multiple types of colonies that are different types but have similar colors. In this case, even if color features are used in the colony detection systems described in Patent Documents 2 and 3, there is a problem in that it is difficult to identify different types of colonies that have similar color features from images captured by cameras. Note that the identification target is not limited to colonies, and even if the target is of a different type but has a similar color, the same problem occurs.

Below, means for solving the above problems and their effects will be described.
An imaging device that solves the above problems is an imaging device that images a subject and outputs an image that can identify regions made of two different materials included in the subject, and the imaging device that captures the subject is RGB. an imaging unit having pixels, a monochromatic light source that illuminates the subject with monochromatic light, a control unit that controls the monochromatic light source and the imaging unit, and a three-band RGB image output from the imaging unit that captures the image of the subject. an image processing unit that performs predetermined processing on the RGB pixel, and the RGB pixel includes a first pixel having sensitivity in a first wavelength region and a second pixel having sensitivity in a second wavelength region, The wavelength range and the second wavelength range overlap in some overlapping wavelength ranges, and the monochromatic light has an emission wavelength range that includes at least a part of the first wavelength range and at least a part of the overlapping wavelength range. The image processing unit includes a calculation processing unit that generates a three-band XYZ image with RGB imaging characteristics corrected by performing linear matrix calculation processing on the three-band RGB image, and the monochromatic light source. The XYZ image is constructed with a gain set to a value that allows each output level of the first pixel and the second pixel to be balanced between the two colors under the condition that monochromatic light is irradiated from the An amplifying section that amplifies each value corresponding to each of the first pixel and the second pixel among the XYZ values.

According to this configuration, the imaging device can output an image in which regions made of two different materials included in the subject can be identified by color or density. For example, when an image is used for inspection, two different types of materials included in the image can be identified or classified with relatively high accuracy. Note that in this specification, "density" is a concept equivalent to pixel level, signal level, pixel value, etc.

In the imaging device, the image processing unit may generate the three-band XYZ image by performing black balance processing on a reference portion of the subject other than the identification target.
According to this configuration, since black balance processing is performed, even if the environment when photographing a subject changes, it is possible to output an image in which the material can be identified with appropriate color or density.

In the imaging device, the monochromatic light source is a blue light source or a red light source, and when the monochromatic light source is the blue light source, the first pixel is a B pixel, and the second pixel is a G pixel, When the monochromatic light source is the red light source, the first pixel may be an R pixel, and the second pixel may be a G pixel.

According to this configuration, the G pixel has sensitivity to at least part of the overlapping wavelength region also to blue light, which is the light source color of the blue light source. Therefore, when capturing an image by illuminating with blue light, the sensitivity wavelength range of the G image can be narrowed compared to that of a general-purpose camera. Furthermore, the G pixel has sensitivity to at least a portion of the overlapping wavelength region for red light, which is the light source color of the red light source. Therefore, when capturing an image by illuminating with red light, the sensitivity wavelength range of the G image can be narrowed compared to that of a general-purpose camera. By performing linear matrix calculation on the RGB image, it is possible to further narrow the narrow sensitive wavelength region of the G image or shift the wavelength. Therefore, it is possible to output an image in which the material can be identified with appropriate color or density.

In the above imaging device, when the monochromatic light source is a first monochromatic light source, a second monochromatic light source capable of emitting a second monochromatic light having an emission wavelength range different from an emission wavelength range of the first monochromatic light emitted by the first monochromatic light source The RGB pixel further includes a light source, the RGB pixel includes a third pixel in addition to the first pixel and the second pixel, and the control unit simultaneously performs illumination with the first monochromatic light source and the second monochromatic light source. a gain to which the gain is set when the subject is imaged by the imaging unit and the amplification unit amplifies each value corresponding to each of the first pixel and the second pixel among the XYZ values; a setting section, wherein the control section illuminates the second monochromatic light source at a second illuminance set according to the illuminance of the first monochromatic light source; or The gain setting section may set a gain to be given to the output level of the third pixel according to each gain given to the output level of the second pixel.

According to this configuration, by adjusting the illuminance of the monochromatic light source or setting the gain, it is possible to output an image in which the material can be identified with an appropriate color or density.
An inspection apparatus for solving the above problem is an inspection apparatus equipped with the above-mentioned imaging device, wherein the object includes two different types of colonies as the two different types of materials, and the three bands inputted from the imaging device are The identification unit includes an identification unit that identifies colony areas that are areas of two different colonies based on an XYZ image, the XYZ image includes an X image, a Y image, and a Z image, and the identification unit The area whose level is within the threshold setting range is defined as a first colony area, the area whose pixel level is within the threshold setting range in the Y image is defined as a second colony area, and the pixel level is within the threshold setting range in the Z image. The area is a third colony area, the remaining colony area after excluding the second colony area from the first colony area or the third colony area is a fourth colony area, and the first colony area and the third colony are a classification unit that classifies at least one of the areas, the second colony area, and the fourth colony area based on characteristics and shapes; and counting the number of each of the colony areas classified by the classification unit. and a specifying section that specifies the number of two types of colony regions to be identified based on the counting results of the counting section.

According to this configuration, by adjusting the illuminance of the monochromatic light source or setting the gain, it is possible to output an image in which the material can be identified with an appropriate color or density. Therefore, when using an image for inspection, two different types of materials included in the image can be identified or classified with relatively high accuracy.

A method for determining imaging conditions for solving the above problem is a method for determining imaging conditions for an imaging device that outputs an image that can identify regions made of two different materials included in a subject, the imaging device It is equipped with a monochromatic light source that illuminates with monochromatic light, and an imaging section that includes RGB pixels that images the subject, and performs linear matrix calculation processing on the three-band RGB image output from the imaging section to obtain RGB imaging characteristics. a first step of setting a matrix coefficient by which the RGB image is multiplied in the linear matrix calculation process in order to generate a three-band XYZ image corrected; The emission wavelength of the monochromatic light source is changed so that the peak wavelength of the spectral output characteristic of one pixel and the spectral output characteristic of a second pixel different from the first pixel is changed, and the monochromatic light source after the emission wavelength change is changed. a second step of illuminating the subject; a third step of causing the imaging unit to image the subject illuminated with the monochromatic light in the second step; and an output from the imaging unit as a result of the imaging in the third step. a fourth step of multiplying the RGB image by the matrix to generate an XYZ image; and a second image output from the second pixel of the XYZ image generated in the fourth step, which is different from the two types. a fifth step of determining whether the difference in density between regions of the material is equal to or greater than a predetermined threshold; and in the fifth step, the first The first to fifth steps are repeated with at least one of a change in the coefficient in the step and a change in the emission wavelength in the second step.

According to this method for determining imaging conditions, it is possible to determine appropriate imaging conditions that enable the imaging device to output an image in which regions made of two different materials included in the subject can be distinguished by color or density.

An imaging method that solves the above problems is an imaging method that images a subject and outputs an image that can identify regions made of two different materials included in the subject, the imaging method comprising: an imaging unit that captures the subject; a monochromatic light source that illuminates the subject with monochromatic light; the imaging unit includes an imaging element having RGB pixels; the RGB pixels include a first pixel having sensitivity in a first wavelength region; and a monochromatic light source having sensitivity in a first wavelength region; a second pixel having sensitivity to , the first wavelength region and the second wavelength region overlap in some overlapping wavelength regions, and the monochromatic light has at least a portion of the first wavelength region and the second wavelength region. The light is in an emission wavelength range including at least a part of the overlapping wavelength range, and includes an imaging step in which the imaging unit takes an image of the subject illuminated by the monochromatic light source, and 3 output from the imaging unit that has taken the image of the subject. a calculation processing step of generating a three-band XYZ image with corrected RGB imaging characteristics by performing linear matrix calculation processing on the band RGB image; Using each gain set to color balance the output level of the first pixel and the second pixel between the two colors, two pixels corresponding to the first pixel and the second pixel among the three bands of XYZ images are obtained. and an amplification step of amplifying the image of the species.

According to this imaging method, it is possible to output an image in which regions made of two different materials included in the subject can be identified by color or density. For example, by using an image output by this method, two different types of materials included in the image can be identified or classified with relatively high accuracy.

According to the present invention, it is possible to output an image in which regions made of two different materials included in a subject can be identified by color or density.

It is a schematic side view showing an inspection device provided with an imaging device in an embodiment. FIG. 3 is a schematic plan view showing a subject. 1 is a schematic diagram showing a schematic configuration of a camera, and a graph showing relative sensitivity of the camera for each RGB color. It is a partial schematic diagram which shows the detailed structure of a monochromatic light source. FIG. 2 is a block diagram showing the functional configuration of the inspection device. 7 is a graph illustrating a case where it is difficult to identify different types of imaging targets. 7 is a graph illustrating the principle of making it easier to identify different types of imaging targets. 12 is a graph illustrating a case where it is difficult to identify different types of imaging targets in images captured using a general-purpose camera. It is a graph showing the density of white colonies and yellow colonies in an RGB image. It is a graph showing the relationship between the wavelength of a light source and the output power (light intensity). It is a graph showing the relationship between the wavelength and relative output of an RGB image sensor when illuminated with blue and red monochromatic light. It is a graph showing the relationship between wavelength and relative output after linear matrix calculation processing. 7 is a graph illustrating the principle by which output images of an imaging device captured using a general-purpose camera identify different types of imaging targets. It is a graph showing the density of white colonies and yellow colonies in an XYZ image. (a) is an XYZ image, (b) is an X image, (c) is a Y image, and (d) is a diagram showing a Z image. 3 is a flowchart showing an initial setting routine. 3 is a flowchart showing an imaging processing routine. 3 is a flowchart showing an inspection processing routine. 3 is a flowchart showing an imaging condition determination processing routine. It is a graph showing the relationship between the wavelength of the light source and the output power (light intensity) in a modified example. It is a graph showing the relationship between wavelength and relative output after linear matrix calculation processing when illuminating with green and red monochromatic light in a modified example. It is a graph explaining the principle by which the output image of the imaging device imaged using the general-purpose camera in a modification example identifies different types of imaging targets.

Hereinafter, an imaging device and an inspection device according to embodiments will be described with reference to the drawings.
This embodiment is an example in which an imaging device 11 that captures an image of a subject 12 is included in an inspection apparatus 10 that inspects the subject 12. Note that the imaging device 11 may be included in a device other than the inspection device 10, or may be used alone.

<Inspection equipment>
As shown in FIG. 1, the inspection device 10 includes an imaging device 11 that images the subject 12, and an inspection processing unit 60 that inspects the subject 12 using images input from the imaging device 11. The inspection device 10 may include a display section 70 that displays the inspection results of the inspection processing section 60 and the like. The display unit 70 may display an image output from the imaging device 11, an image obtained by processing the image, or the like. The display unit 70 may be, for example, a monitor connected to a personal computer, or a display provided on an operation panel included in the inspection device 10.

The imaging device 11 includes a monochromatic light source 20 that illuminates a subject 12 with monochromatic light, an imaging unit 30 that captures an image of the subject 12, a control unit 40 that controls the monochromatic light source 20 and the imaging unit 30, and an imaging unit 30 that outputs an image as an imaging result. The image processing unit 50 includes an image processing unit 50 that performs predetermined processing on the output image signal. The imaging device 11 also includes a power source 45 that supplies power to the monochromatic light source 20 . The imaging unit 30 is, for example, a color camera 31. The color camera 31 (hereinafter also simply referred to as "camera 31") is electrically connected to the processing section 18. The processing section 18 includes the aforementioned control section 40 and image processing section 50. The processing unit 18 may include at least one of an electronic circuit and a computer.

The monochromatic light source 20 may be one type of monochromatic light source that emits one type of monochromatic light, or may include two types of monochromatic light sources that emit two types of monochromatic light. In this case, the monochromatic light source 20 may include three types of monochromatic light sources that emit three types of monochromatic light. When a plurality of types of monochromatic light sources are provided, the control unit 40 may be configured to cause one or two types of monochromatic light sources to emit light. Here, one type of monochromatic light source refers to any one of a blue light source, a red light source, and a green light source.

The monochromatic light source is composed of, for example, an LED. The monochromatic light source may be, for example, any one of a blue LED, a red LED, and a green LED. The monochromatic light source 20 of this example includes at least a blue LED and a red LED. Therefore, in the example shown in FIG. 1, the power supply 45 includes at least a blue LED power supply 46 and a red LED power supply 47.

The control unit 40 includes an imaging control unit 41 that controls the imaging unit 30 and an illumination control unit 42 that controls the monochromatic light source 20. The illumination control unit 42 controls turning on and off of the monochromatic light source 20 via a power source 45. When the monochromatic light source 20 includes a plurality of types of monochromatic light sources (LEDs), the lighting control unit 42 individually controls turning on and off for each monochromatic light source via the power source 45. Therefore, the subject 12 is illuminated with monochromatic light by the monochromatic light source 20. Note that the monochromatic light source 20 may illuminate the subject 12 with two types of monochromatic light. In short, all three types of monochromatic light sources (blue light source, red light source, green light source) must simultaneously emit three types of monochromatic light (blue light source, red light source, green light source) to illuminate the subject 12 with white light. For example, the subject 12 may be illuminated with two types of monochromatic light appropriately selected from among blue light, red light, and green light. Furthermore, the inspection device 10 may include a robot that transports objects to be transported, such as a petri dish 13 as the subject 12 and a reference sheet for color gain adjustment, to a predetermined position. In this case, the control unit 40 may control the robot, or a control unit different from the control unit 40 may control the robot.

The image processing section 50 performs predetermined image processing on the first image signal S1 of three RGB bands output from the camera 31 constituting the image capturing section 30, and generates a second image signal S2 of three XYZ bands (both shown in FIG. 5). reference). Note that RGB in the first image signal S1 refers to three colors determined from each wavelength range that can be received by the R pixel, G pixel, and B pixel that constitute the image sensor 33 of the camera 31. Furthermore, the XYZ referred to in the second imaging signal S2 is determined from each wavelength region of the XYZ image obtained by converting the RGB image forming the first imaging signal S1 into an XYZ image having each wavelength region different from each wavelength region of RGB. Refers to three colors.

The inspection processing unit 60 inspects the subject 12 using the XYZ image based on the second imaging signal S2 input from the image processing unit 50.
As shown in FIGS. 1 and 2, the subject 12 includes colonies WC and YC cultured in a Petri dish 13. That is, the subject 12 includes a petri dish 13, a medium 14 in the petri dish 13, and colonies WC and YC cultured in the medium 14. Note that colonies WC and YC do not occur at the beginning of the culture period or when there are no bacteria, so colonies WC and YC may not exist depending on the subject 12.

In the examples shown in FIGS. 1 and 2, two or more different types of colonies WC and YC exist. In the examples shown in FIGS. 1 and 2, the different types of colonies WC and YC are two types having similar colors. Furthermore, foreign matter FS as shown in FIG. 2 may exist in the culture medium 14. For example, when inspecting food or drinking water for the presence of bacteria, the foreign substances FS include dietary fiber, food particles, seeds, skin, fruit pulp, etc. contained in the food or drinking water. Foreign matter FS may be mistakenly identified as colonies WC and YC. Therefore, the inspection processing unit 60 identifies the colonies WC, YC and the foreign object FS based on their characteristics. For example, colonies WC and YC are identified as foreign matter FS based on characteristic factors such as shape, area, and color. Colonies WC and YC generally have a circular or oval shape, and the size is within a predetermined range. When the foreign material FS is a fiber, its shape is elongated. Further, when the foreign substance FS is fruit pulp, seeds, skin, etc., the shape is different from the colonies WC and YC. For example, if the aspect ratio of the circumscribed rectangle is used as the feature factor of the shape, if the aspect ratio is within a predetermined range close to 1, it can be identified as a colony WC or YC, and if it is outside the predetermined range, it can be identified as a foreign object FS. . In addition, the inspection processing unit 60 distinguishes colony candidates from colonies WC, YC and others based on a threshold value of area (size).

As shown in FIGS. 1 and 2, the petri dish 13 is placed on a background sheet 15 (background board). The background sheet 15 serves as a background for the petri dish 13, which is the subject 12. The background sheet 15 may be, for example, a black diffusion plate. By making the background sheet 15 black, it may be used as a black reference for black balance. The petri dish 13 is made of a colorless and transparent material (for example, glass). The culture medium 14 contained in the petri dish 13 is colored as necessary. In that case, the culture medium 14 may be colored in a predetermined color and become translucent. In the image captured by the camera 31, the black color of the background sheet 15 may be transmitted through the culture medium 14. Furthermore, when testing food or drinking water, the culture medium 14 may be colored depending on the color of the food or drinking water. Note that the black background sheet 15 may be a black plate such as a black sheet that does not have a diffusion function of diffusing light. Further, when using the background sheet 15 as a reference for black balance, it is not limited to a black (achromatic color) sheet, but a chromatic color sheet may be used.

The two types of colonies schematically shown in FIGS. 1 and 2 are white colony WC and yellow colony YC. The white colony WC and the yellow colony YC are different types of colonies. Therefore, for inspection, it is necessary to distinguish between white colony WC and yellow colony YC. However, since the white and yellow colors of the white colony WC and the yellow colony YC are similar in color, it is difficult to distinguish them in the RGB image captured by the camera 31 of the subject 12. Further, in the R image, G image, and B image that constitute the RGB image, the white colony WC and the yellow colony YC have similar densities, so it is difficult to distinguish them. In this way, in the RGB image obtained by imaging the subject 12 with the camera 31, it is difficult to distinguish between the white colony WC and the yellow colony YC. In this embodiment, the image processing unit 50 performs predetermined image processing on the first imaging signal S1 from the camera 31, thereby generating an XYZ image that makes it easy to distinguish between the white colony WC and the yellow colony YC. Details of the XYZ image will be described later.

<Configuration of camera 31>
Next, with reference to FIG. 3, the internal structure of the camera 31 that constitutes the imaging section 30 will be described. In FIG. 3, the camera 31 is a general-purpose color camera that captures RGB images. The camera 31 includes a lens 32 assembled to a lens barrel 31a, a near-infrared light cut filter 35 (hereinafter also referred to as IR cut filter 35) that blocks near-infrared light, and an image sensor 33.

The image sensor 33 includes an R pixel 33R, a G pixel 33G, and a B pixel 33B, each of which is a light receiving element. The R pixel 33R receives red light (R light) that has passed through the R filter 34R, and outputs an R pixel signal according to the amount of received light. The G pixel 33G receives green light (G light) that has passed through the G filter 34G, and outputs a G pixel signal according to the amount of received light. The B pixel 33B receives blue light (B light) that has passed through the B filter 34B, and outputs a B pixel signal according to the amount of received light. In the image sensor 33, an R pixel 33R, a G pixel 33G, and a B pixel 33B are arranged in a predetermined arrangement.

This image sensor 33 has RGB imaging characteristics in which near-infrared light is cut. The R pixel 33R, the G pixel 33G, and the B pixel 33B are sensitive to light in respective wavelength bands shown in the graph in FIG. In the graph, the horizontal axis is wavelength and the vertical axis is relative sensitivity. The R pixel 33R has high sensitivity to light in the red (R) wavelength band shown in the graph in FIG. The G pixel 33G has high sensitivity to light in the green (G) wavelength band shown in the graph. The B pixel 33B has high sensitivity to light in the blue (B) wavelength band shown in the graph.

<About the configuration of the monochromatic light source 20>
Next, the configuration of the monochromatic light source 20 will be described with reference to FIG. 4.
As shown in FIG. 4, the monochromatic light source 20 includes monochromatic light sources of three colors (three types): a blue light source 21, a green light source 22, and a red light source 23. The blue light source 21 emits blue light (B light) as monochromatic light. The green light source 22 emits green light (G light) as monochromatic light. The red light source 23 emits red light (R light) as monochromatic light. When capturing an image of the subject 12, the illumination control unit 42 of this embodiment illuminates the subject 12 with blue light (B light) by causing the blue light source 21 to emit light as a monochromatic light source. Note that if the emitted light color (light source color) is not particularly conscious, the blue light source 21, green light source 22, and red light source 23 that constitute the monochromatic light source 20 may be referred to as monochromatic light sources 21 to 23.

Here, the monochromatic light source 20 may have the function of a white light source. That is, the monochromatic light source 20 also has a white light source function of emitting white light by emitting three types of monochromatic light sources 21 to 23 at the same time. In this case, the control unit 40 causes the white light source to emit any two types (two colors) of the three types of monochromatic light sources 21 to 23 capable of emitting monochromatic light of three colors of RGB, so that the white light source can emit monochromatic light. Function as a light source. In this specification, a white light source having the function of such a monochromatic light source is also included in the monochromatic light source 20. Note that the first imaging in which the subject 12 is imaged by illuminating with one type or two types of monochromatic light, and the second imaging in which the subject 12 is imaged by illuminating with white light may be performed in a time-sharing manner. In this case, images captured using white light may also be used for the inspection.

The control unit 40 determines which monochromatic light source emits light based on the spectral reflectance characteristics of the colony to be inspected. Here, one type of two types of monochromatic light sources that are emitted simultaneously when photographing the subject 12 is assumed to be a first monochromatic light source 25 which is a main monochromatic light source, and the other type is assumed to be a second monochromatic light source 26 which is an auxiliary light source. . In this example, as shown in FIG. 4, the blue light source 21 is used as a first monochromatic light source 25, and the red light source 23 is used as a second monochromatic light source 26. That is, the blue light source 21 is the main first monochromatic light source 25, and the first monochromatic light source 25 has a first monochromatic light source 25 that has an emission wavelength range different from the emission wavelength range of the first monochromatic light (B light in this example) that illuminates the subject 12. The red light source 23 capable of emitting two monochromatic lights (R light in this example) is assumed to be a second monochromatic light source 26 which is an auxiliary light source. The control unit 40 illuminates the subject 12 using blue light (B light) as main monochromatic light and illuminates the subject 12 using red light (R light) as auxiliary light. Which two of the three RGB colors are used as a combination of the first monochromatic light source 25 and the second monochromatic light source 26 is determined according to the spectral reflectance characteristics of the object to be inspected.

The monochromatic light sources 21 to 23 are, for example, LEDs. In this example, the blue light source 21 is a blue LED, the green light source 22 is a green LED, and the red light source 23 is a red LED. The control unit 40 is capable of individually and independently controlling on/off the three types of monochromatic light sources 21 to 23. When imaging the subject 12, in the example shown in FIG. 1, the control unit 40 uses the blue LED determined from the imaging condition search results performed in advance to identify the white colonies WC and yellow colonies YC to be inspected. In addition to this, a red LED is lit as an auxiliary light. Note that a method for determining imaging conditions including a method for determining monochromatic light used for illumination will be described later.

<Configuration of image processing unit 50 and inspection processing unit 60>
Next, detailed configurations of the image processing section 50 and the inspection processing section 60 will be described with reference to FIG. 5.
As shown in FIG. 5, an image of the subject 12 is formed on the imaging surface of the image sensor 33 through the lens 32 and the IR cut filter 35. The image sensor 33 outputs the first imaging signal S1 as the imaging result of the subject 12 to the image processing unit 50. The first imaging signal S1 is a serial signal including an R imaging signal (red signal), a G imaging signal (green signal), and a B imaging signal (blue signal) from each pixel 33R, 33G, and 33B. Note that the R imaging signal, the G imaging signal, and the B imaging signal are also simply referred to as an R signal, a G signal, and a B signal.

As shown in FIG. 5, the image processing section 50 includes an RGB separation section 51, an arithmetic processing section 52, and an amplification section 53. The RGB separation unit 51 separates the first image signal S1 input from the image sensor 33 into an R image signal, a G image signal, and a B image signal.

The arithmetic processing unit 52 converts the R signal, G signal, and B signal input from the RGB separation unit 51 into an X signal, a Y signal, and a Z signal. Specifically, the arithmetic processing unit 52 converts the RGB values, which are the signal values of the R signal, G signal, and B signal, into the X signal, Y signal, and Z signal by performing a linear matrix calculation. The arithmetic processing unit 52 is given matrix coefficients. Here, the matrix used for the linear matrix calculation is a 3×3 matrix. The arithmetic processing unit 52 is given coefficients of a 3×3 matrix.

The arithmetic processing unit 52 performs a linear matrix operation of multiplying the RGB values of the first imaging signal S1 by a 3×3 matrix specified using the matrix coefficient, and calculates the spectral characteristics different from the RGB values of the first imaging signal S1. It is converted into a second image pickup signal S2 expressed by XYZ with . The matrix coefficient is a coefficient for separating RGB of the first image signal S1 into XYZ bands of the second image signal S2.

Here, the calculation formula for converting the RGB signal, which is the first imaging signal S1, into the XYZ signal, which is the second imaging signal S2, is given by the following equation (1).

Here, a1 to a3, b1 to b3, and c1 to c3 are matrix coefficients, and Gx, Gy, and Gz are gains (amplification factors). These matrix coefficients are set in the matrix coefficient setting section 54 shown in FIG. Further, the gains Gx, Gy, and Gz are set in a gain setting section 55 shown in FIG. The gain setting unit 55 sets the output levels of the first and second pixels of the RGB pixels of the image sensor 33 between two colors under the condition that the first monochromatic light is irradiated from the first monochromatic light source 25. A gain value that allows for color balance is set. In the example shown in FIG. 12, the gain setting unit 55 is set to the B pixel 33B, which is the first pixel, under the condition that B light, which is the first monochromatic light, is irradiated from the blue light source 21, which is the first monochromatic light source 25. A gain is set to a value that allows each output level of the second pixel G pixel 33G to be balanced between two colors (B color and G color).

The arithmetic processing unit 52 performs a linear matrix arithmetic process of multiplying the RGB values by a 3×3 matrix in the above equation (1). The arithmetic processing unit 52 outputs the XYZ values before being multiplied by a gain (amplification factor) to the amplification unit 53. Matrix coefficients are set in the 3×3 matrix that can improve the separation of the three bands.

For example, when the blue light source 21 and the red light source 23, which are monochromatic light sources shown in FIG. is given. That is, as coefficients of the 3×3 matrix, a1=2, a2=-0.04, a3=0, b1=-0.55, b2=1, b3=-0.25, c1=0, c2=- 0.15, c3=1 is given. These matrix coefficients are set in the matrix coefficient setting section 54 shown in FIG. This is just an example, and other matrix coefficients may be set.

The amplification unit 53 multiplies the XYZ values before normalization from the arithmetic processing unit 52 by the X gain Gx, Y gain Gy, and Z gain Gz set in the gain setting unit 55, respectively. The amplification unit 53 multiplies the X value after XYZ conversion by the X gain Gx, the Y value by the Y gain Gy, and the Z value by the Z gain Gz. The amplification unit 53 outputs the normalized XYZ values as the second image pickup signal S2. In this example, the amplification unit 53 is capable of balancing the output levels of the B pixel 33B and the G pixel 33G between two colors under the condition that monochromatic light (B light) is irradiated from the blue light source 21. set to a value. In this way, the image processing unit 50 sequentially performs RGB separation processing, linear matrix calculation processing (XYZ conversion processing), and normalization processing on the input first imaging signal S1, thereby outputting a second imaging signal S2. . In this way, the three-band RGB image signal is converted into the three-band XYZ image signal.

<Configuration of inspection processing unit 60>
Next, the configuration of the inspection processing section 60 will be explained with reference to FIG. 5.
The inspection processing section 60 includes an identification section 61 and a determination section 62. The identification section 61 includes a classification section 63, a counting section 64, and a specifying section 65.

The subject 12 includes two different types of colonies WC and YC as two different types of materials. The identification unit 61 identifies colony areas, which are areas of two different types of colonies WC and YC, based on the three-band XYZ image input from the imaging device 11. The XYZ image includes an X image, a Y image, and a Z image. The identification unit 61 defines an area in which the pixel values constituting the X image are within the range set by the first threshold value as a first colony area. Further, the identification unit 61 defines an area in which the pixel values forming the Y image are within the range set by the second threshold value as a second colony area. Furthermore, the identification unit 61 defines an area where the pixel values forming the Z image are within the range set by the third threshold value as a third colony area. The identification unit 61 removes the second colony area from the first colony area or the third colony area and sets the remaining colony area as a fourth colony area. For example, the first colony area or the third colony area is an area where all of the colonies WC and YC are in the subject 12. The second colony region is a region of white colony WC or yellow colony YC. The calculation formula for the fourth colony region described here is an example of the case of FIG. 12, and the calculation formula for the fourth colony region changes depending on the characteristics of the colony to be identified. Note that the first to third thresholds are thresholds for setting a specific range of pixel values, and not only one type of intermediate threshold, but also two types of thresholds, an upper threshold and a lower threshold, or two intervals. Four types of threshold values may be used to set the range.

The identification unit 61 binarizes the area specified within the threshold range and the other areas using the threshold value, and specifies the area to be inspected extracted by the binarization.
The classification unit 63 classifies at least one of the first colony area and the third colony area, the second colony area, and the fourth colony area based on characteristics such as shape and size (area).

The counting unit 64 counts the number of each colony area classified by the classification unit 63.
The specifying unit 65 specifies the number of two types of colony areas to be identified based on the counting results of the counting unit 64.

The determination unit 62 determines the quality of the test object obtained as a result of culturing colonies in the Petri dish 13 of the subject 12 based on the counts of the two types of colony regions. In addition, when the inspection device 10 is a colony counter, it may be provided with a determination function for determining the quality of the inspection object as an auxiliary function, or may be configured without the determination function by the determination section 62. Furthermore, when the inspection target of the inspection device 10 is not a colony, the main function may be a determination function for determining the quality of the inspection target.

<Light reflection characteristics of the colony to be inspected and sensitivity of the imaging device (image sensor)>
Next, with reference to FIG. 6, the reason why it is difficult to distinguish the two types of colonies WC and YC with similar colors even if they are imaged using a general-purpose color camera will be explained. Further, with reference to FIG. 7, in this embodiment, even in images captured using the same general-purpose color camera 31, two types of colonies WC and YC can be distinguished by performing predetermined processing on the imaging signal. I will explain about it.

FIG. 6 shows an example of spectral reflectance characteristic lines LW and LY of white colony WC and yellow colony YC, and spectral sensitivity characteristic line B indicating the spectral sensitivity characteristic of B pixel as an example of RGB pixels. In the graph of FIG. 6, the horizontal axis shows wavelength (nm), and the vertical axis shows relative sensitivity. Note that the spectral reflectance characteristic lines LW and LY of the two types of colonies shown in this graph are hypothetical waveforms for convenience of explanation. Generally, a dedicated spectral reflectance measurement device is required to measure the spectral reflectance characteristics of an inspection target such as a colony.

The spectral reflectance characteristics of white colony WC and yellow colony YC are similar. Further, the culture medium 14 in the petri dish 13 may be colored in a predetermined color. Therefore, due to the coloring of the medium 14, it becomes even more difficult to distinguish between the white colony WC and the yellow colony YC.

In FIG. 6, the spectral reflectance characteristic line LW of the white colony WC and the spectral reflectance characteristic line LY of the yellow colony YC are similar over a wide wavelength range. A spectral sensitivity characteristic line B of the B pixel shown by a broken line in FIG. 6 indicates the relative sensitivity of the B pixel, and has a peak near 430 nm. The spectral reflectance characteristic lines LW and LY of each colony WC and YC both have peaks that overlap with the spectral sensitivity characteristic line B. Furthermore, both have a peak in a predetermined wavelength region of the near-infrared region NIRA.

The B pixel 33B is sensitive to B light (light in the blue to green region), which is a light component in the range of 300 to 600 nm, as shown by the broken line in FIG. Therefore, the B pixel 33B receives the blue component of the B light of the reflected light from the white colony. Further, the B pixel 33B receives the blue component of the B light of the reflected light from the yellow colony.

At this time, the density of the white colony WC in the B image is proportional to the area Sw of the region LW*B multiplied by both the spectral reflectance characteristic line LW and the spectral sensitivity characteristic line B of the white colony WC in FIG. . Further, the density of the yellow colony YC in the B image is proportional to the area Sy of the region LY*B, which is obtained by multiplying both the spectral reflectance characteristic line LY and the spectral sensitivity characteristic line B of the yellow colony YC in FIG. 6. In this case, the two areas Sw and Sy have approximately the same value. Therefore, the densities of the two types of colonies WC and YC in the B image are approximately the same. That is, in the B image, the two types of colonies WC and YC are difficult to distinguish. Similarly, in the G and R images, there is not much difference in density between the two types of colonies WC and YC. In both the G image and the R image, the two types of colonies WC and YC are difficult to distinguish.

On the other hand, as shown in FIG. 7, B is limited to a narrow wavelength range (400 to 500 nm) in which the difference between the two spectral reflectance characteristic lines LW and LY of white colony WC and yellow colony YC is relatively large. If the spectral sensitivity characteristics of the pixels exist, white colonies WC and yellow colonies YC can be distinguished in the B image. That is, the density of the white colony WC in the B image is proportional to the area Sw of the region LW*B multiplied by both the spectral reflectance characteristic line LW and the spectral sensitivity characteristic line B of the white colony WC in FIG. 7. Further, the density of the yellow colony YC in the B image is proportional to the area Sy of the region LY*B, which is obtained by multiplying both the spectral reflectance characteristic line LY and the spectral sensitivity characteristic line B of the yellow colony YC in FIG. 7. In this case, since there is a difference between the two areas Sw and Sy by a predetermined value (predetermined ratio) or more, the two types of colonies WC and YC can be identified in the B image.

Similarly, in FIG. 7, the narrow broken line N is limited to the near-infrared wavelength region (800 to 900 nm) where the difference between the two spectral reflectance characteristic lines LW and LY of the white colony WC and the yellow colony YC is relatively large. With the spectral sensitivity characteristics shown, the two types of colonies WC and YC can be distinguished in the IR image. In addition, in FIGS. 6 and 7, Sw indicates the area of the area where LW and B overlap, not LW*B, but since the difference from LW*B is small, in these figures, Sw indicates LW*B. is substituted by the area of the region where LW and B overlap. Similarly, for Sy, LY*B is replaced by the area of the region where LY and B overlap.

In this way, by narrowing the width (wavelength range) of the spectral sensitivity characteristic of the pixel or shifting the position (wavelength) of the spectral sensitivity characteristic, the spectral reflectance characteristic lines LW, Spectral sensitivity characteristics of pixels are set in areas where there is a difference in LY. In this way, the two types of colonies WC and YC can be identified from the image. This requires a multispectral or hyperspectral camera. Alternatively, a special light source that can emit light of a specific wavelength with a narrow emission wavelength range is required. However, these special cameras and special light sources are expensive, which increases the manufacturing cost of the imaging device.

In this embodiment, the spectral sensitivity characteristics of pixels as shown in FIG. 7 are generated by performing predetermined processing on the first image signal S1 captured using a general-purpose color camera 31.
<About images using general-purpose color camera 31>
Next, a comparative example will be described with reference to FIG.

As shown in FIG. 8, the image sensor 33 mounted on the general-purpose color camera 31 has the RGB spectral sensitivity characteristics shown in the graph when illuminated with white LED. In this graph, the spectral reflectance characteristic lines LW and LY of the white colony WC and the yellow colony YC are different from those in FIGS. 6 and 7.

In the example shown in FIG. 8, there is a difference between the spectral reflectance characteristic line LW of the white colony WC and the spectral reflectance characteristic line LY of the yellow colony YC in the region DA where the wavelength is about 480 to 520 nm. In the area DA where this difference exists, if any of RGB has a sensitivity peak in a wavelength range as narrow as the width of this area DA, then in the image of that color, white colonies WC and yellow colonies YC can be distinguished by density. Become.

However, in the general-purpose color camera 31, the spectral sensitivity characteristic lines R, G, and B, which indicate the sensitivity of each of the RGB pixels shown in FIG. 8, are distributed over a wide wavelength range. The spectral reflectance characteristic line LW of the white colony WC and the spectral reflectance characteristic line LY of the yellow colony YC have a difference of LW>LY in the wavelength range of about 480 to 520 nm, but on the contrary, in the wavelength range of about 530 to 580 nm. LW<LY. Therefore, the difference in density between the two types of colonies WC and YC becomes small in any of the R, G, and B images that have sensitivity in the range of 400 to 700 nm.

For example, the density of the white colony WC in the G image is calculated by multiplying the spectral reflectance characteristic line LW (thick line) of the white colony WC in FIG. 8 by the spectral sensitivity characteristic line G of the G pixel shown by the dashed line The area of the region is indicated by Sg1. The density of the yellow colony YC in the G image is calculated by multiplying the spectral reflectance characteristic line LY (thick one-dot chain line) of the yellow colony in FIG. 8 by the spectral sensitivity characteristic line G of the G image, which is the area Sg2 of the region. It is indicated by. The ratio of the difference ΔSg (=|Sg1−Sg2|) between the two areas Sg1 and Sg2 to the whole (total area of the G region) is small. As this ratio becomes smaller, it becomes difficult to distinguish between the two types of colonies WC and YC in the G image. This is because the G pixel has sensitivity over a wide wavelength range. Similarly, in the B and R images, since the differences ΔSb and ΔSr (not shown) account for a small proportion of the total area (total area of B area, total area of R area), white colony WC and yellow colony YC cannot be distinguished. It's hard to do. In FIG. 8, the area Sg1 of the area where LW and G overlap is substituted for LW*G, and the area Sg2 of the area where LY and G overlap is substituted for LY*G.

FIG. 9 shows the density of white colonies WC and yellow colonies YC in an RGB image captured by the general-purpose color camera 31 of this comparative example. In any of the R, G, and B images, there is almost no difference between the density Cw of the white colony WC and the density Cy of the yellow colony YC, or even if there is a difference, it is very small. For this reason, it is difficult to distinguish between white colony WC and yellow colony YC in an RGB image.

As shown in FIG. 8, the RGB pixels of the image sensor 33 of the general-purpose color camera 31 have a sensitivity expressed as a relative output with respect to wavelength (nm) under white LED illumination. The B pixel 33B, which is the first pixel, has a relative output of 0.1 or more for light having a wavelength of approximately 440 to 540 nm, and has spectral characteristics having a peak at approximately 460 nm. That is, the first wavelength region where the relative output is 0.1 or more is about 440 to 540 nm. Further, the G pixel 33G, which is the second pixel, has a relative output of 0.1 or more for light having a wavelength of approximately 460 to 610 nm, and has a spectral characteristic having a peak at approximately 550 nm. That is, the second wavelength region where the relative output is 0.1 or more is about 460 to 610 nm. The R pixel 33R, which is the third pixel, has a relative output of 0.1 or more for light having a wavelength of approximately 570 to 650 nm, and has spectral characteristics having a peak at approximately 600 nm.

The first wavelength region (approximately 440 to 540 nm) and the second wavelength region (approximately 460 to 610 nm) overlap in some overlapping wavelength regions (approximately 460 to 540 nm). When irradiated with B light whose emission wavelength includes at least part of the overlapping wavelength range of about 460 to 550 nm, not only the first B pixel 33B but also the second G pixel 33G also has a relative output of 0.1 or more.

FIG. 10 shows the spectrum (emission distribution) of the monochromatic light source 20 when the blue light source 21 made of a blue LED and the red light source 23 made of a red LED emit light at the same time. In the graph, the horizontal axis is wavelength and the vertical axis is output power. The blue light source 21, which is the main monochromatic light source, emits B light of about 430 to 530 nm. The red light source 23, which is an auxiliary light source, emits R light of 500 to 660 nm. Note that in the graph of FIG. 10, the spectral characteristics of the white LED are shown by broken lines.

FIG. 11 shows the relative outputs of the RGB pixels 33R, 33G, and 33B when illuminated with B light and R light. As shown in FIG. 11, the G region is shifted to the lower wavelength side compared to the G region when illuminated with white light in FIG. Furthermore, in region B shown in FIG. 11, the region where the relative output was about 0.05 to 2 in the region of 500 to 650 nm in region B when illuminated with white light in FIG. 8 has almost disappeared.

Further, as shown in FIG. 11, the R region is shifted to the higher wavelength side by illuminating with the R light shown in FIG. 10, compared to the R region when illuminating with white light in FIG.
Then, as shown in FIG. 12, an XYZ image is obtained by performing linear matrix calculation on the RGB image.

The imaging device 11 of this example employs an imaging method that allows a color camera 31 equipped with an RGB color filter 34 to have new two-band imaging characteristics through monochromatic illumination and RGB linear matrix calculation processing. . The two new bands have different peak wavelengths from the original two bands, and have different imaging characteristics from the RGB imaging characteristics when using white LED lighting. In addition, by using monochromatic illumination in a region different from the new two-band sensitivity wavelength region as an auxiliary light source, this imaging method enables color discrimination that is difficult to perform when imaging with a normal color camera 31.

The monochromatic illumination of this embodiment is blue illumination. The monochromatic illumination used as an auxiliary light source is red illumination. That is, the blue light source 21 that illuminates with B light is used as the main monochromatic light source, and the red light source 23 that illuminates with R light is used as an auxiliary light source. By illuminating with R light using the red light source 23 as an auxiliary light source, the R image is also used for inspection processing.

Since the overlapping wavelength region is about 460 to 550 nm, as shown in FIG. 11, the B light from the blue light source 21 causes the G pixel 33G to have a relative output of 0.1 or more in the region of about 450 to 520 nm. The B light, which is monochromatic light that illuminates the subject 12, is light in an emission wavelength range that includes at least part of the first wavelength range and at least part of the overlapping wavelength range.

The control unit 40 simultaneously illuminates the first monochromatic light source 25 and the second monochromatic light source 26 to cause the imaging unit 30 to image the subject 12. The control unit 40 illuminates the second monochromatic light source 26 with a second illuminance set according to the illuminance of the first monochromatic light source 25, or illuminates the B pixel which is the first pixel among the RGB pixels 33R, 33G, and 33B. The gain setting unit 55 sets the gain given to the output level of the R pixel 33R, which is the third pixel, according to each gain given to the output level of the R pixel 33B, which is the third pixel, and the G pixel 33G, which is the second pixel.

As shown in FIG. 13, the density of the white colony WC in the Y image is determined by the spectral reflectance characteristic line LW (thick line) of the white colony WC in FIG. The area of the region calculated by multiplication is indicated by Sg1. In addition, the density of the yellow colony YC in the Y image is calculated by multiplying the spectral reflectance characteristic line LY (thick one-dot chain line) of the yellow colony in FIG. 13 by the spectral sensitivity characteristic line Y of the Y image, the area Sg2 of the region. It is indicated by. The difference ΔSg (=|Sg1−Sg2|) between the two areas Sg1 and Sg2 accounts for a large proportion of the total (sum of Sg1 and Sg2). Since this ratio corresponds to the density difference ratio, in the Y image, the two types of colonies WC and YC have a large density difference ratio and are easy to distinguish. Note that between the B image and the R image, the difference ΔSb and ΔSr (not shown) account for a small proportion of the total area (the total area of the B region, the total area of the R region), so the two types of colonies WC and YC cannot be distinguished. Hateful. In addition, in FIG. 13, the area Sg1 of the area where LW and Y overlap is substituted for LW*Y, and the area Sg2 of the area where LY and Y overlap is substituted for LY*Y.

FIG. 14 shows the density of white colonies WC and yellow colonies YC in an XYZ image captured by the general-purpose color camera 31 of this embodiment. In the X image and the Z image, there is almost no difference between the density Cw of the white colony WC and the density Cy of the yellow colony YC, or even if there is a difference, the difference is very small. For this reason, it is difficult to distinguish between the two types of colonies WC and YC in the X/Z images. However, in the Y image, there is a large difference between the density Cw of the white colony WC and the density Cy of the yellow colony YC. Therefore, white colonies WC and yellow colonies YC can be easily distinguished in the Y image. From the above, it is also possible to identify the two types of colonies WC and YC by color in the XYZ image.

<Contents of control by the control unit 40>
<Action of the embodiment>
Next, the functions of the imaging device 11 and the inspection device 10 will be explained.

<About initial setting process>
First, the initial setting process will be described with reference to FIG. 16. The control unit 40 executes the initial setting processing routine shown in FIG.

In step S11, the control unit 40 installs a reference sheet for color gain adjustment. The control unit 40 controls, for example, a robot to have an arm grip a reference sheet, and places the gripped reference sheet at, for example, an inspection position, which is an imaging area of the camera 31 . Note that the reference sheet may be placed manually by the operator.

In step S12, the control unit 40 turns on the blue LED.
In step S13, the control unit 40 sets the B and G gains so that the levels of the B and G images are the same. That is, the control unit 40 sets the respective output levels of the B pixel 33B, which is the first pixel, and the G pixel 33G, which is the second pixel, to 2 under the condition that the B pixel 33B, which is the first pixel, is illuminated with B light from the blue LED, which is the blue light source 21. The B gain and G gain are set to values that allow for color balance between colors.

In step S14, the control unit 40 turns on the red LED.
In step S15, the control unit 40 manually sets the R gain. That is, since the operator manually sets the R gain by operating the input section, the control section 40 sets the R gain input from the input section by writing it into a predetermined storage area of the memory.

In step S16, the control unit 40 causes a background sheet to be installed. The control unit 40 controls, for example, the arm of the transport robot, causes the arm to grip the background sheet, and places the gripped background sheet at the imaging area of the camera 31, for example, at an inspection position. Note that before placing the background sheet at the inspection position, the robot removes the reference sheet previously placed at the inspection position.

In step S17, the control unit 40 calculates and sets a black balance offset amount using the background sheet image.
<About imaging processing>
Next, the imaging process will be described with reference to FIG. 17. The control unit 40 executes the imaging processing routine shown in FIG. 17. The control unit 40 controls the camera 31 to take an image of the petri dish 13 placed at the inspection position, and also processes the image processing unit 50 (FIGS. 1 and 5) with respect to the imaging signal output from the camera 31 as an imaging result. ) performs predetermined image processing to convert the 3-band RGB image forming the imaging signal into a 3-band XYZ image and output it.

Specifically, in step S21, the control unit 40 arranges the petri dish 13 in which the colonies WC and YC are cultured on the background sheet 15. In this process, the control unit 40 controls, for example, a robot to place the petri dish 13 held by the arm on the background sheet 15. Note that the operator may place the petri dish 13 on the background sheet 15.

In the next step S22, the control unit 40 turns on the blue and red LEDs.
In step S23, the control unit 40 causes the camera 31 to take an image. Note that the processing in steps S22 and S23 corresponds to an example of an imaging step.

In step S24, the control unit 40 performs linear matrix processing on the RGB image. Note that the processing in step S24 corresponds to an example of an arithmetic processing step.
In step S25, the control unit 40 performs white balance processing using the gains Gx, Gy, and Gz set in the gain setting unit 55. Here, the gains Gx and Gy are values obtained by color-balancing the output levels of the B pixel and the G pixel. Therefore, even if the areas Sg1 and Sg2 shown in FIG. 13 occupy a small area in the entire Y area, the two types of colonies WC and YC do not become too dark in the Y image of FIG. It can be obtained with a concentration difference.

In step S26, the control unit 40 performs black balance processing using the set RGB offset amount. That is, the control unit 40 performs black balance so that the black color of the background sheet 15 in the image has the same density as the black color of the background sheet 15 in the image set at the time of initial setting.

In step S27, the control unit 40 outputs an XYZ image.
As shown in FIG. 15(a), in the XYZ image, a distinguishable difference in density is obtained between the white colony WC and the yellow colony YC by predetermined image processing such as binarization processing. Therefore, two types of colonies WC and YC can be identified. On the other hand, in the X image shown in FIG. 15(b), the concentrations of the two types of colonies WC and YC are the same and difficult to distinguish, but the difference in concentration between the two types of colonies WC and YC is large; It is possible to identify Therefore, the total number of colonies WC and YC can be counted. Further, in the Y image shown in FIG. 15(c), a distinguishable difference in density is obtained between the white colony WC and the yellow colony YC. Therefore, two types of colonies WC and YC can be identified. In addition, in the Z image shown in FIG. 15(d), the concentrations of the two types of colonies WC and YC are about the same and difficult to distinguish, but since the difference in concentration between the two types of colonies WC and YC is large, there is a large concentration difference between the two types of colonies WC and YC. It is possible to identify The total number of colonies WC and YC can be counted. Furthermore, the relationship between the concentrations of the culture medium 14 and the colonies WC and YC is different between the X image and the Y image. Therefore, depending on how the culture medium 14 is colored, by using both the X image and the Z image, it is possible to count the total number of colonies WC and YC using at least one of these images.

Note that the areas of the two types of colonies WC and YC in FIG. 15(b) correspond to the first colony area, which is an area where the pixel level is within the threshold setting range. One of the two types of colonies WC and YC in FIG. 15(c) corresponds to the second colony area, which is an area where the pixel level is within the threshold setting range. The areas of the two types of colonies WC and YC in FIG. 15(d) correspond to the third colony area, which is an area where the pixel level is within the threshold setting range. Then, the remaining colony area after excluding the second colony area, which is the area of one of the two types of colonies, from the first colony area or the third colony area, which is the area of the total number of colonies, corresponds to the fourth colony area. do. For example, if the white colony WC area corresponds to the second colony area, the yellow colony YC area corresponds to the fourth colony area, and if the yellow colony YC area corresponds to the second colony area, the white colony WC area corresponds to the fourth colony area. Corresponds to the fourth colony area.

<About inspection processing>
Next, inspection processing will be described with reference to FIG. 18. The computer constituting the inspection processing section 60 executes the inspection processing routine shown in FIG. The inspection processing unit 60 counts colonies for each image using the XYZ images from the imaging device 11, and performs a counting correction (count correction) that corrects the number of colonies by type using the colony counting result for each image. A determination process is performed to determine the test result based on the corrected counting result (count result).

The processing performed for each XYZ image includes area extraction processing to find the colony area to be inspected, classification processing to classify the extracted area according to the characteristics of the colony, and counting the number of inspection subjects classified as colonies. This includes counting processing (counting processing). Specifically, the area extraction process is a process of finding the area of the inspection target (colony candidate) for each of the X image, Y image, and Z image. The classification process is a process of classifying whether or not a colony to be inspected exists in the extracted region, using the shape factor and size factor of the colony to be inspected as characteristics. The counting process is a process of counting the number of test objects classified as colonies.

Then, by performing a predetermined calculation using multiple colony count values (count values) obtained for each XYZ image, and performing count correction to correct the number for each type of colony, the count for each type of colony can be calculated. Find the numerical value. Further, determination processing such as determining the quality of the inspection object using the count values for each type of colony is performed to obtain a final inspection result. Note that FIG. 18 shows parallel processing for determining regions for each XYZ image in one flow, and the processing order of step numbers is shown as an example, but the parallel processing may be performed in any order. The feature classification and counting for each area may be performed in any order as long as the area is specified.

In FIG. 18, steps S31 to S34 are a series of processes (first process) for counting colonies using the X image. Steps S35 to S38 are a series of processes (second process) for counting colonies using the Y image. Steps S39 to S42 are a series of processes (third process) for counting colonies using the Z image. Steps S43 to S47 are a series of processes (fourth process) in which two or three of the areas extracted as colony candidates in the first to third processes are combined to determine a specific type of colony area. .

The first process includes region extraction using a threshold value of the X image (step S31), identification of a first region as a region extraction result (step S32), feature classification for classifying the first region according to features (step S33), and feature It includes the count of classified inspection objects (step S34).

The second process includes region extraction using a threshold value of the Y image (step S35), identification of a second region as a region extraction result (step S36), feature classification for classifying the second region according to the feature (step S37), and feature classification of the second region according to the feature. It includes the count of classified inspection objects (step S38).

The third process includes region extraction using a threshold value of the Z image (step S39), identification of a third region as a region extraction result (step S40), feature classification for classifying the third region according to features (step S41), and feature It includes the count of classified inspection objects (step S42).

The fourth process is a process of finding another area using the multiple areas obtained in each of the above processes (for example, three processes). The fourth process includes a determination process (step S43) that determines whether a plurality of areas to be used have already been acquired, and an area acquisition process (step S43) that determines a fourth area using the first area and the second area, as an example. S44), identification of a fourth region as a result of region acquisition (step S45), feature classification for classifying the fourth region according to features (step S46), and counting of test objects subjected to feature classification (step S47). Note that in this example, the colony candidate area of the A fourth colony area may be determined by using the colony candidate area of the Y image as the second area.

In step S48, the inspection processing unit 60 (more specifically, the computer configuring the inspection processing unit 60) performs a predetermined calculation using the count results of the inspection targets for each region obtained in steps S34, S38, S42, and S47, respectively. Perform count correction. Count values for each type of colony to be inspected, for example, are determined by count correction. For example, the number of colonies of the second type is calculated by subtracting the number of colonies of the first type from the total number of colonies. Further, in step S49, the inspection processing unit 60 performs a determination process to obtain a final inspection result using the count correction results. Judgment processing includes inspection judgment processing that determines whether the total number of colonies and the number of each type of colony exceeds the corresponding number threshold, and highlighting colonies by surrounding them with a display frame or adding a predetermined color. It may also include highlighting processing and the like. The test processing section 60 then displays the test results on the display section 70. The display unit 70 displays, for example, the total number of colonies, the number of colonies by type, test result details, and images with colonies highlighted. In addition, when the inspection target is food or drinking water, the inspection result content may include a determination result as to whether the product is good or defective. Moreover, although the inspection target is a colony here, it is not limited to this. For example, a configuration may be adopted in which foreign matter contained in a product is inspected for the presence of foreign matter, or a configuration in which contamination on the product is inspected for the presence or absence of contamination may be adopted.

In the present embodiment, the identification section 61 and the determination section 62 are configured by software by the computer forming the inspection processing section 60 executing the inspection processing program shown in FIG. Then, the identification unit 61 identifies each area by area extraction (steps S31, S35, S39, S44) and area specification (steps S32, S36, S40, S45). The classification unit 63 is configured by a computer that performs feature classification processing (steps S33, S37, S41, and S46). The counting unit 64 is constituted by a computer that performs counting processing (steps S34, S38, S42, S47). The specifying unit 65 is constituted by a computer that performs a process of specifying the number of two types of colony areas to be identified based on the count value that is the counting result of the counting unit 64. The determination unit 62 is constituted by a computer that performs determination processing based on the numbers of the two types of colonies identified (step S49).

<About imaging condition determination process>
Next, with reference to FIG. 19, the imaging condition determination process will be described. In this example, the control unit 40 executes the imaging condition determination processing routine shown in FIG. 19. This flowchart is an example of searching and determining imaging conditions using three types of monochromatic light sources. Note that the types of monochromatic light sources used for the imaging condition search may be two types of monochromatic light sources among the blue light source 21, the green light source 22, and the red light source 23.

In FIG. 19, the processing in steps S51 to S57 is a first search process in which imaging conditions are searched and determined using the first monochromatic light source, which is the first monochromatic light source. If appropriate imaging conditions cannot be determined even after repeatedly performing the processes in steps S51 to S57, the monochromatic light is changed in step S58, and the processes in steps S51 to S57 are repeatedly performed. Modification of monochromatic light refers to modification of the peak wavelength of monochromatic light. Furthermore, if all the peak wavelengths of monochromatic light in one monochromatic light source have been changed, the monochromatic light source may be changed. Even if the monochromatic light source is changed, the processes in steps S51 to S57 are basically the same. Therefore, in the following, a process in which the control unit 40 searches for and determines imaging conditions using a blue light source, which is the first monochromatic light source, will be described in detail.

First, in step S51, the control unit 40 sets coefficients for linear matrix calculation when illuminating with the first monochromatic light. This step S51 performs linear matrix calculation processing on the 3-band RGB image output from the imaging unit 30 to generate a 3-band XYZ image with corrected RGB imaging characteristics. This corresponds to an example of the first step of setting matrix coefficients by which the RGB image is multiplied.

In step S52, the control unit 40 illuminates the subject with the first monochromatic light source. In this step S52, the subject 12 is photographed with monochromatic light that can change the peak wavelengths of the spectral output characteristics of the first pixel among the RGB pixels of the camera 31 and the spectral output characteristics of the second pixel different from the first pixel. This corresponds to an example of the second step of illuminating.

In step S53, the control unit 40 images the subject. Note that the process in step S53 corresponds to an example of a third step in which the imaging unit 30 images the subject 12 illuminated with monochromatic light in the second step.

In step S54, the control unit 40 multiplies the RGB image by a matrix to generate an XYZ image. Note that this step S53 corresponds to an example of the fourth step in which the RGB image output from the imaging unit 30 as a result of the imaging in the third step is multiplied by a matrix to generate an XYZ image.

In step S55, the control unit 40 obtains the difference in density between two areas of different types in the Y image. Note that this step S55 determines whether the difference in density between regions of two different materials in the second image output from the second pixel of the XYZ image generated in the fourth step is greater than or equal to a predetermined threshold. This corresponds to an example of the fifth step of determining whether.

In step S56, the control unit 40 determines whether the difference is greater than or equal to a threshold value. If the difference is not greater than or equal to the threshold, that is, if the difference is less than the threshold, the process advances to step S57. On the other hand, if the difference is greater than or equal to the threshold, the process proceeds to step S59, where the monochromatic light and coefficients at that time are determined. Note that the determined coefficients are set in the matrix coefficient setting section 54.

In step S57, the control unit 40 determines whether the condition search using the first monochromatic light has ended. If the condition search using the first monochromatic light has not been completed, the process returns to step S52 and the processes from step S52 to step S57 are repeatedly executed. If an imaging condition for which the difference is equal to or greater than the threshold value cannot be found even after completing all the condition searches using the first monochromatic light, the process moves to step S58.

In step S58, the control unit 40 changes the monochromatic light. Specifically, the control unit 40 changes the peak wavelength of monochromatic light. Here, changing the peak wavelength of monochromatic light may mean gradually changing the peak wavelength of the monochromatic light source without changing it. For example, the peak wavelength of a general-purpose blue LED is about 470 nm, but it may be changed to 480 nm and 490 nm in 10 nm steps, for example. As a means for this, there are two methods: leaving the monochromatic light source (LED) as it is and shifting the peak wavelength using an optical filter, or using another monochromatic light source with a different peak wavelength of the monochromatic light source (LED) itself.

In this way, the peak wavelength of the monochromatic light (for example, B light) to be used is changed, and the imaging condition search processing in steps S51 to S57 is similarly executed using the monochromatic light after the peak wavelength has been changed. If an imaging condition for which the difference is still equal to or greater than the threshold cannot be found in step S56, the monochromatic light source to be used is changed from the blue light source 21 to the green light source 22 in step S58. Then, similarly, the peak wavelength of the monochromatic light (for example, G light) to be used is changed, and the imaging condition search processing of steps S51 to S57 is similarly executed using the monochromatic light after the peak wavelength has been changed. If an imaging condition for which the difference is still equal to or greater than the threshold cannot be found in step S56, the green light source 22 is changed to the red light source 23 in step S58. Then, similarly, the peak wavelength of the monochromatic light (for example, R light) to be used is changed, and the imaging condition search processing of steps S51 to S57 is similarly executed using the monochromatic light after the peak wavelength change. In this way, the change in the coefficient in the first step (S51) and the peak of monochromatic light in the second step (S52, S58) are performed until the difference in density becomes equal to or greater than a predetermined threshold in the third step (S56). The first to fifth steps (steps S51 to S58) are repeated as the wavelength changes. Note that when the monochromatic light source is changed in step S58, the target image when acquiring the density difference among the XYZ images in the subsequent step S55 is an image different from the previous Y image (X image or Z image). ) may be changed. This image may be selected depending on the monochromatic light and the spectral reflectance characteristics of the colonies WC and YC.

The imaging device 11 images the subject 12 under the imaging conditions determined in this routine. In this example, when an object 12 consisting of a petri dish 13 containing two types of colonies WC and YC in a culture medium 14 is imaged by illuminating it with a blue light source 21, an imaging condition is found in which the difference is equal to or greater than a threshold value. , the blue light source 21 is determined to be a monochromatic light source. This blue light source 21 is determined as the first monochromatic light source 25.

In the above description, in step S58, changing the peak wavelength of the monochromatic light includes changing the monochromatic light by changing the peak wavelength while leaving the monochromatic light source as it is. It is sufficient if the emission wavelength can be changed so that the peak wavelengths of the first pixel and the spectral output characteristics of the second pixel, which are different from the first pixel, can be changed. For example, it is sufficient to simply change the monochromatic light source. That is, in step S58, the control unit 40 may simply change the blue light source 21, the green light source, and the red light source 23 in order. In addition, in the case of a configuration in which the monochromatic light source is unchanged but the peak wavelength is changed, the type of monochromatic light source used for imaging condition search can be one of the blue light source 21, the green light source 22, and the red light source 23. be.

<Effects of embodiment>
According to the embodiment described in detail above, the following effects can be obtained.
(1) The imaging device 11 outputs an image in which regions made of two different materials included in the photographed subject 12 can be identified. The imaging device 11 includes an imaging section 30, a monochromatic light source 20, a control section 40, and an image processing section 50. In the imaging unit 30, an image sensor 33 (imaging device) that images the subject 12 has RGB pixels 33R, 33G, and 33B. The monochromatic light source 20 illuminates the subject 12 with monochromatic light. The control unit 40 controls the monochromatic light source 20 and the imaging unit 30. The image processing unit 50 performs predetermined processing on the three-band RGB image output from the imaging unit 30 that has captured the image of the subject 12. The RGB pixels include a first pixel 33B having sensitivity in a first wavelength region and a second pixel 33G having sensitivity in a second wavelength region. The first wavelength region and the second wavelength region overlap in some overlapping wavelength regions. Monochromatic light is light in an emission wavelength range that includes at least part of the first wavelength range and at least part of the overlapping wavelength range. The image processing section 50 includes an arithmetic processing section 52 and an amplification section 53. The arithmetic processing unit 52 generates a three-band XYZ image with corrected RGB imaging characteristics by performing linear matrix arithmetic processing on the three-band RGB image. The amplification unit 53 sets each output level of the first pixel 33B and the second pixel 33G to a value that allows color balance between the two colors under the condition that monochromatic light is irradiated from the monochromatic light source 20. The gain amplifies each value corresponding to each of the first pixel 33B and the second pixel 33G among the XYZ values forming the XYZ image.

According to this configuration, by adjusting the illuminance or setting the gain of the monochromatic light source 20, it is possible to output an image in which the materials (colonies WC, YC) can be identified with appropriate color or density. Therefore, when using an image for inspection, two different types of materials included in the image can be identified or classified with relatively high accuracy.

(2) The image processing unit 50 generates a three-band XYZ image by performing black balance processing on a reference portion of the subject 12 other than the identification target. According to this configuration, since black balance processing is performed, even if the environment when photographing the subject 12 changes, it is possible to output an image in which the material can be identified with appropriate color or density.

(3) The monochromatic light source 20 is a blue light source 21 or a red light source 23. When the monochromatic light source 20 is the blue light source 21, the first pixel 33B is a B pixel, and the second pixel is a G pixel 33G. When the monochromatic light source 20 is the red light source 23, the first pixel is the R pixel 33R, and the second pixel is the G pixel 33G.

According to this configuration, the G pixel 33G has sensitivity to at least part of the overlapping wavelength region also to blue light, which is the light source color of the blue light source 21. Therefore, when capturing an image by illuminating with blue light, the sensitivity wavelength range of the G image can be narrowed compared to that of a general-purpose camera. Furthermore, the G pixel 33G has sensitivity to at least a portion of the overlapping wavelength region with respect to red light, which is the light source color of the red light source 23. Therefore, when capturing an image by illuminating with red light, the sensitivity wavelength range of the G image can be narrowed compared to that of the general-purpose camera 31. By performing linear matrix calculation on the RGB image, it is possible to further narrow the narrow sensitive wavelength region of the G image or shift the wavelength. Therefore, it is possible to output an image in which the materials (colonies WC, YC) can be identified with appropriate color or density.

(4) When the blue light source 21 is used as the first monochromatic light source 25, the imaging device 11 can emit second monochromatic light in an emission wavelength range different from that of the first monochromatic light emitted by the first monochromatic light source 25. It further includes a second monochromatic light source 26. The RGB pixels include a third pixel in addition to the first pixel 33B and the second pixel 33G. The control unit 40 causes the imaging unit 30 to image the subject 12 by illuminating the first monochromatic light source 25 and the second monochromatic light source 26 . The control unit 40 illuminates the second monochromatic light source 26 with a second illuminance set according to the illuminance of the first monochromatic light source 25, or controls the output of the first pixel 33B and second pixel 33G among the RGB pixels. The gain setting unit 55 sets the gain given to the output level of the third pixel according to each gain given to the level. According to this configuration, by adjusting the illuminance or setting the gain of the monochromatic light source 20, it is possible to output an image in which the material can be identified with an appropriate color or density.

(5) The inspection device 10 includes an imaging device 11. The object 12 includes two different colonies WC and YC as two different materials. An identification unit 61 is provided that identifies colony areas, which are areas of two different types of colonies WC and YC, based on the three-band XYZ image input from the imaging device 11. The XYZ image includes an X image, a Y image, and a Z image. The identification unit 61 defines an area in the X image whose pixel level is within the threshold setting range as a first colony area. The identification unit 61 defines an area in the Y image whose pixel level is within the threshold setting range as a second colony area. The identification unit 61 defines an area in the Z image whose pixel level is within the threshold setting range as a third colony area. The identification unit 61 removes the second colony area from the first colony area or the third colony area and sets the remaining colony area as a fourth colony area. The identification section 61 includes a classification section 63 , a counting section 64 , and a specifying section 65 . The classification unit 63 classifies at least one of the first colony area and the third colony area, the second colony area, and the fourth colony area based on characteristics and shapes. The counting unit 64 counts the number of each colony area classified by the classification unit. The specifying unit 65 specifies the number of two types of colony areas to be identified based on the counting results of the counting unit. According to this configuration, by adjusting the illuminance or setting the gain of the monochromatic light source 20, it is possible to output an image in which the material can be identified with an appropriate color or density. Therefore, when using an image for inspection, two different types of materials included in the image can be identified or classified with relatively high accuracy.

(6) A method for determining imaging conditions for the imaging device 11 that outputs regions made of two different materials included in the subject 12 as images that can be identified. The imaging device 11 includes a monochromatic light source 20 that illuminates the subject 12 and an imaging unit 30 that includes RGB pixels that captures an image of the subject 12. This imaging condition determining method includes first to fifth steps. The first step is to perform linear matrix calculation processing on the 3-band RGB image output from the imaging unit 30 to generate a 3-band XYZ image with corrected RGB imaging characteristics. The coefficients of the matrix by which the RGB image is multiplied are set. In the second step, it is possible to change the peak wavelengths of the spectral output characteristics of the first pixel 33B and the spectral output characteristics of the second pixel 33G, which is different from the first pixel 33B, among the RGB pixels forming the imaging unit 30. The subject 12 is illuminated with monochromatic light. In the third step, the imaging unit 30 images the subject 12 illuminated with monochromatic light in the second step. In the fourth step, the RGB image output from the imaging unit 30 as a result of the imaging in the third step is multiplied by a matrix to generate an XYZ image. The fifth step is to determine whether the difference in density between regions of two different materials in the second image output from the second pixel 33G among the XYZ images generated in the fourth step is greater than or equal to a predetermined threshold. to judge. Until the difference in concentration becomes equal to or greater than a predetermined threshold value in the fifth step, the first to second steps are performed with at least one of a change in the coefficient in the first step and a change in each peak wavelength in the second step. Repeat the 5 steps. According to this method of determining imaging conditions, appropriate imaging conditions are set under which the imaging device 11 can output an image in which regions made of two different materials (colonies WC, YC) included in the subject 12 can be identified by color or density. can be determined.

(7) This is an imaging method that outputs regions made of two different materials included in the imaged subject 12 as images that can be identified. It includes an imaging unit 30 that images the subject 12, and a monochromatic light source 20 that illuminates the subject 12 with monochromatic light. The imaging unit 30 includes an image sensor 33 (image sensor) having RGB pixels. The RGB pixels include a first pixel 33B having sensitivity in a first wavelength region and a second pixel 33G having sensitivity in a second wavelength region. The first wavelength region and the second wavelength region overlap in some overlapping wavelength regions. Monochromatic light is light in an emission wavelength range that includes at least part of the first wavelength range and at least part of the overlapping wavelength range. This imaging method includes an imaging step, an output step, an arithmetic processing step, and an amplification step. In the imaging step, the imaging unit 30 images the subject 12 illuminated by the monochromatic light source 20 . In the output step, the imaging unit 30 that has captured the image of the subject 12 outputs a 3-band RGB image. The arithmetic processing step generates a three-band XYZ image with corrected RGB imaging characteristics by performing linear matrix arithmetic processing on the three-band RGB image output from the imaging unit 30. The amplification step is performed in three bands using each gain set to balance the output levels of the first pixel 33B and the second pixel 33G between two colors under the condition that monochromatic light is irradiated from the monochromatic light source 20. Among the XYZ images, two types of images corresponding to the first pixel 33B and the second pixel 33G are amplified. According to this imaging method, it is possible to output an image in which regions made of two different types of materials (colonies WC, YC) included in the subject 12 can be identified by color or density. For example, when an image is used for inspection, two different types of materials included in the image can be identified or classified with relatively high accuracy.

The embodiment is not limited to the above, and may be modified in the following manner.
- Monochromatic light is not limited to blue light. Further, the combination of the first monochromatic light and the second monochromatic light is not limited to blue light and red light. For example, as shown in FIG. 20, the monochromatic light emitted by the monochromatic light source 20 may be green light or red light. That is, the first monochromatic light may be green light, and the combination of the first monochromatic light and the second monochromatic light may be green light and red light. In this case, by performing linear matrix calculation on the RGB image, an XYZ image as shown in FIG. 21 is obtained. Then, as shown in FIG. 22, in the area DA, the difference ΔSg between the area Sg1 and the area Sg2 becomes relatively large. Therefore, a relatively large difference occurs in the ratio of the difference ΔSg to the total area Sg1 and the area Sg2, making it easier to distinguish between white colonies and yellow colonies in the XYZ image (particularly the B image) based on the density difference between the two.

- In FIG. 3, the IR cut filter 35 that blocks near-infrared light from the camera 31 may be removed, and the RGB pixels forming the image sensor may be configured to have sensitivity to near-infrared light. The first monochromatic light source 25 is the blue light source 21, and B light is used as the first monochromatic light. The second monochromatic light source 26 is a near-infrared LED, and near-infrared light is used as the second monochromatic light. As shown in FIG. 7, a linear matrix operation is performed on an RGB image to convert, for example, a NYZ image containing N images having a peak in the near-infrared region (for example, approximately 800 to 900 nm in the example of FIG. 7) to an XYZ image. Generate as an image. In this case, in the wavelength region that overlaps with this near-infrared region, there is a difference between the lines LW and LY that indicate the spectral characteristics of the white colony WC and the yellow colony YC, so in the NYZ image (especially the N image), the white colony WC and the yellow colony It becomes easier to distinguish between YC and YC. In this way, one or two of the three colors XYZ may have a wavelength region within the near-infrared region.

- The inspection processing unit 60 may perform inspection processing using only two images, the Y image and the Z image, or only the two images, the X image and the Y image.
- The monochromatic light source 20 may include a near-infrared light source that uses near-infrared light as monochromatic light as one type of monochromatic light source.

- The B light is not limited to monochromatic light emitted by a blue LED, but may be other blue light having a peak in the blue wavelength region.
- In the embodiment, when the main monochromatic light is B light, the first pixel is the B pixel and the second pixel is the G pixel, but the present invention is not limited to this. When the monochromatic light is R light, the first pixel may be the R pixel 33R, and the second pixel may be the G pixel 33G. Further, when the monochromatic light is G light, the first pixel may be the G pixel 33G, and the second pixel may be the B pixel 33B or the R pixel 33R.

- The combination of the first monochromatic light and the second monochromatic light is not limited to blue light and red light as in the above embodiment. Moreover, it is not limited to green light and red light as shown in FIG. The monochromatic light may be red light. In this case, the first monochromatic light may be red light and the second monochromatic light may be blue light.

・The number of monochromatic light sources is not limited to two types, and only one type may be used. In the embodiment, the monochromatic light source may be only a blue light source. Further, in FIG. 20, the monochromatic light source may be only a green light source.

- In the case of a color camera that has color filters of primary colors, the four colors of R, G1, G2, and B may be used. Furthermore, a color camera having complementary color filters may be used, and the complementary colors may be four colors: yellow, cyan, magenta, and green.

- Save image data (for example, RGB image data) based on the first imaging signal S1 captured by the image sensor 33 using the camera 31 in a removable memory such as a USB memory. The image data stored in the removable memory may be read by a personal computer, and the CPU (image processing unit 50) of the personal computer may perform conversion processing including linear matrix calculation to generate an XYZ image. In other words, the device that performs the imaging step and the device that performs the conversion step may be separate devices. Also by such an imaging method, XYZ images of multiple bands can be acquired.

- The configuration may be such that an inspector visually recognizes and inspects the XYZ image output by the imaging device 11.
- Similar colors are not limited to white and yellow, but may also include white and light pink, white and light orange, white and light green, white and light blue, etc. In addition, a combination of white and light color may be used. Alternatively, a combination of different light colors may be used.

- The object 12 is not limited to one that includes colonies, such as a Petri dish in which colonies are cultured, but may be an object that includes different types of identification targets. Further, the object to be identified may be a transparent object or a semi-transparent object. Furthermore, the identification target may be a food such as jelly or a processed food.

・The subject 12 to be imaged or inspected is not particularly limited. The object 12 may be, for example, a container such as a plastic bottle or a bottle, a food product, a beverage, an electronic component, an electric appliance, a daily necessities, a part, a member, or a raw material such as powder, granule, or liquid. In addition, the inspection target may be scratches, dirt, printing defects, painting defects, etc. Furthermore, the object within the subject 12 is not limited to the inspection object. The objects may be of different types but exhibit similar colors.

- The imaging device 11 may be configured as a separate device from the inspection processing section 60. The imaging device 11 may be used for purposes other than inspection.
- The arrangement pattern of the color filters 34 constituting the image sensor 33 is not limited to the RGB Bayer arrangement, but may be any arrangement pattern such as a stripe arrangement.

- The imaging device 11 does not need to include a transport robot. For example, a configuration may be adopted in which an operator places the subject 12 on a mounting table for photographing, which serves as an imaging position.
- At least one of the control unit 40, the image processing unit 50, and the inspection processing unit 60 may be partially or entirely configured by software consisting of a computer that executes a program, or may be configured by hardware such as an electronic circuit. may be configured.

DESCRIPTION OF SYMBOLS 10... Inspection device, 11... Imaging device, 12... Subject, 13... Petri dish, 14... Culture medium, 15... Background sheet, 18... Processing part, 20... Monochromatic light source, 21... Blue light source (blue light source) constituting an example of a monochromatic light source LED), 22... Green light source (green LED) constituting an example of a monochromatic light source, 23... Red light source (red LED) constituting an example of a monochromatic light source, 25... First monochromatic light source, 26... Second monochromatic light source, 30 ...Imaging unit, 31...Color camera (camera), 31a...Lens barrel, 32...Lens, 33...Color image sensor (image sensor), 35...Infrared light cut filter (IR cut filter), 33R...R pixel, 33G ...G pixel as an example of a second pixel, 33B...B pixel as an example of a first pixel, 34...color filter, 34R...R filter, 34G...G filter, 34B...B filter, 40...control unit, 41... Imaging control unit, 42... Illumination control unit, 45... Power supply, 46... Blue LED power supply, 47... Red LED power supply, 50... Image processing unit, 51... RGB separation unit, 52... Arithmetic processing unit, 53... Amplification unit, 54 ...Matrix coefficient setting section, 55... Gain setting section, 60... Inspection processing section, 61... Identification section, 62... Judgment section, 63... Classification section, 64... Counting section, 65... Specification section, 70... Display section, WC... White colony, YC...yellow colony, S1...first imaging signal, S2...second imaging signal, VA...visible light wavelength region, NIRA...near infrared wavelength region, LW...spectral reflectance characteristic line of white colony, LY... Spectral reflectance characteristic line of yellow colony, B, G, R...spectral sensitivity characteristic line, X, Y, Z...spectral sensitivity characteristic line calculated from RGB spectral sensitivity characteristic, Gx, Gy, Gz...gain.

Claims (7)

1. An imaging device that images a subject and outputs an image that can identify areas made of two different materials included in the subject,
   an imaging unit in which an imaging device for imaging the subject has RGB pixels;
   a monochromatic light source that illuminates the subject with monochromatic light;
   a control unit that controls the monochromatic light source and the imaging unit;
   an image processing unit that performs predetermined processing on a three-band RGB image output from the imaging unit that captured the image of the subject;
   Equipped with
   The RGB pixel includes a first pixel having sensitivity in a first wavelength region and a second pixel having sensitivity in a second wavelength region,
   The first wavelength region and the second wavelength region overlap in some overlapping wavelength regions,
   The monochromatic light is light in an emission wavelength range including at least part of the first wavelength range and at least part of the overlapping wavelength range,
   The image processing unit includes:
   a calculation processing unit that generates a 3-band XYZ image with corrected RGB imaging characteristics by performing linear matrix calculation processing on the 3-band RGB image;
   The XYZ image is produced with a gain set to a value that allows each output level of the first pixel and the second pixel to be balanced between two colors under the condition that monochromatic light is irradiated from the monochromatic light source. An imaging device comprising: an amplification unit that amplifies each value corresponding to each of the first pixel and the second pixel among the XYZ values forming the image.
2. The imaging device according to claim 1, wherein the image processing unit generates the three-band XYZ image in which black balance processing is performed on a reference portion of the subject other than the identification target.
3. The monochromatic light source is a blue light source or a red light source,
   When the monochromatic light source is the blue light source, the first pixel is a B pixel, and the second pixel is a G pixel,
   The imaging device according to claim 1, wherein when the monochromatic light source is the red light source, the first pixel is an R pixel and the second pixel is a G pixel.
4. If the monochromatic light source is a first monochromatic light source,
   Further comprising a second monochromatic light source capable of emitting a second monochromatic light having an emission wavelength range different from the emission wavelength range of the first monochromatic light emitted by the first monochromatic light source,
   The RGB pixels include a third pixel in addition to the first pixel and the second pixel,
   The control unit causes the imaging unit to image the subject by simultaneously illuminating the first monochromatic light source and the second monochromatic light source,
   a gain setting section in which the gain is set when the amplification section amplifies each value corresponding to each of the first pixel and the second pixel among the XYZ values,
   The control unit may illuminate the second monochromatic light source at a second illuminance set according to the illuminance of the first monochromatic light source, or may illuminate the first pixel and the second pixel among the RGB pixels. 4. The imaging device according to claim 3, wherein the gain setting unit sets the gain given to the output level of the third pixel according to each gain given to the output level.
5. An inspection device comprising the imaging device according to any one of claims 1 to 4,
   The subject includes two different colonies as the two different materials,
   an identification unit that identifies colony areas that are areas of two different types of colonies based on the three-band XYZ image input from the imaging device;
   The XYZ image includes an X image, a Y image, and a Z image,
   The identification section is
   The area in the X image whose pixel level is within the threshold setting range is defined as a first colony area,
   An area in the Y image whose pixel level is within a threshold setting range is defined as a second colony area,
   An area in the Z image whose pixel level is within a threshold setting range is defined as a third colony area,
   The remaining colony area after excluding the second colony area from the first colony area or the third colony area is defined as a fourth colony area,
   a classification unit that classifies at least one of the first colony area and the third colony area, the second colony area, and the fourth colony area based on characteristics and shapes;
   a counting unit that counts the number of each of the colony areas classified by the classification unit;
   An inspection device comprising: a specifying section that specifies the number of two types of colony regions to be identified based on the counting results of the counting section.
6. A method for determining imaging conditions for an imaging device that outputs an image that can identify regions made of two different materials included in a subject, the method comprising:
   The imaging device includes a monochromatic light source that illuminates the subject with monochromatic light, and an imaging unit that captures an image of the subject, including RGB pixels,
   In order to generate a 3-band XYZ image with corrected RGB imaging characteristics by performing linear matrix calculation processing on the 3-band RGB image output from the imaging unit, the linear matrix calculation processing is performed on the RGB image. A first step of setting a coefficient of a matrix to be multiplied by
   The emission wavelength of the monochromatic light source is changed so that the peak wavelength of a spectral output characteristic of a first pixel and a spectral output characteristic of a second pixel different from the first pixel among the RGB pixels constituting the imaging section changes. a second step of illuminating the subject with the monochromatic light source after the emission wavelength has changed;
   a third step in which the imaging unit images the subject illuminated with the monochromatic light in the second step;
   a fourth step of multiplying the RGB image output from the imaging unit as a result of the imaging in the third step by the matrix to generate an XYZ image;
   Determine whether or not the difference in density between regions of the two different materials in the second image output from the second pixel of the XYZ image generated in the fourth step is equal to or greater than a predetermined threshold. Perform the fifth step of
   At least one of the change in the coefficient in the first step and the change in the emission wavelength in the second step is performed until the difference in concentration becomes equal to or greater than a predetermined threshold in the fifth step. A method for determining imaging conditions, characterized in that the first to fifth steps are repeatedly performed.
7. An imaging method that images a subject and outputs an area made of two different materials included in the subject as an identifiable image, the method comprising:
   comprising an imaging unit that images the subject, and a monochromatic light source that illuminates the subject with monochromatic light,
   The imaging unit includes an imaging element having RGB pixels,
   The RGB pixel includes a first pixel having sensitivity in a first wavelength region and a second pixel having sensitivity in a second wavelength region,
   The first wavelength region and the second wavelength region overlap in some overlapping wavelength regions,
   The monochromatic light is light in an emission wavelength range including at least part of the first wavelength range and at least part of the overlapping wavelength range,
   an imaging step in which the imaging unit takes an image of the subject illuminated by the monochromatic light source;
   a calculation processing step of generating a 3-band XYZ image with corrected RGB imaging characteristics by performing linear matrix calculation processing on the 3-band RGB image output from the imaging unit that has imaged the subject;
   The three-band XYZ image is created using each gain set to color balance the output levels of the first pixel and the second pixel between two colors under the condition that the monochromatic light is irradiated from the monochromatic light source. An imaging method comprising: amplifying two types of images corresponding to the first pixel and the second pixel.

Images