JP2018021894A - Image generation apparatus and image generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a compact image generation apparatus.SOLUTION: An image generation apparatus 10 includes: a fist light source 141A and a second light source 141B illuminating a substance; an image sensor 150 on which the subject is placed; a mask 142 having a light-transmitting part where light is transmitted and a light-blocking part where light is blocked, and arranged between the image sensor 150 and the first and second light sources 141A, 141B; and a bright and dark image processing unit 120. The image sensor 150 acquires a first image of the substance when illuminated by the first light sources 141A, and acquires a second image of the subject when illuminated by the second light source 141B. The bright and dark image processing unit 120 derives a difference between a luminance value of a pixel included in the first image and a luminance value of a pixel included in the second image and located in the same position as the pixel included in the first image, to generate a third image of the substance.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to an image generation apparatus and an image generation method for generating an image of, for example, a lensless microscope.

  The demand for continuous observation of cultured cells without staining is in many fields such as production of therapeutic cells or testing of drug efficacy, or the use of cultured cells in medicine or industry. However, since most of the cells are almost colorless and transparent, observation with an optical microscope using transmitted light is difficult because the contrast is small. One factor of the low contrast is light scattering or refraction by the medium around the subject and the subject itself.

  Patent Document 1 shows a method of removing a noise component of reflected light from two brightness levels of light beam irradiation and irradiation stop.

  By the way, continuous observation of cultured cells is performed in a limited space called an incubator for maintaining a humid environment for culturing cells. For observation in such a limited space, Patent Documents 2 and 3 disclose a lensless microscope that can observe minute cells without using a lens. In these documents, a high-resolution image is generated using a plurality of images photographed by illumination irradiated from a plurality of different positions. Non-Patent Document 1 discloses a scattered light removal technique using high-frequency illumination.

Japanese Patent No. 5403458 Japanese Patent No. 5789766 US Patent Application Publication No. 2014/0133702

IEEE International Conference on Computational Photography (ICCP2013) Apr. 2013, Cambridge ‘Descattering of transmissive observation using parallel high-frequency illumination’

  However, the method of Patent Document 1 has a problem that the apparatus becomes large. That is, in this method, parallel light and a digital micromirror device are used in an apparatus for recording the brightness of reflected light of illumination and measuring the unevenness of the surface of the object. Then, the relative luminance is obtained from the difference between the luminance recorded by irradiating the first light / dark pattern and the luminance recorded by irradiating the second light / dark pattern, thereby comparing the luminance at each position of the objective lens. It is easy. However, if the light / dark pattern is to be spread over the entire image sensor, a lens is required and the apparatus becomes large. On the other hand, when photographing with multiple light sources as in Patent Documents 2 and 3, it is necessary to change the position of illumination. In order to perform photographing while changing the positions of the light source and the digital micromirror device, the apparatus becomes large, and it is difficult to combine the reversal of light and darkness by the digital micromirror device with the lensless microscope.

  Therefore, the present disclosure provides an image generation apparatus and an image generation method that can be reduced in size.

  An image generation apparatus according to an aspect of the present disclosure is an image generation apparatus that generates an image of a light-transmitting substance, and includes a first light source that illuminates the substance, and a predetermined distance from the first light source. A second light source that illuminates the substance from a position that is far away, an image sensor in which the substance is disposed, a translucent part through which light from the first light source and the second light source passes, and the light The image sensor includes a light shielding portion for shielding, a mask positioned between the image sensor and the first light source and the second light source, and a processing circuit. The image sensor is illuminated by the first light source. A first image of the substance is obtained when illuminated, and a second image of the substance is obtained when illuminated by the second light source, and the processing circuit applies the first image to the first image. The luminance value of the included pixel and the second image A Murrell pixel, by deriving a difference between the luminance values of the pixels at the same positions as the pixels included in the first image to generate a third image of said material.

  An image generation apparatus according to an aspect of the present disclosure is an image generation apparatus that generates an image of a light-transmitting substance, and includes a first light source that illuminates the substance, and a predetermined amount from the first light source. A light source group including a plurality of light sources including a second light source that illuminates the substance from a position separated by a distance from the image sensor, an image sensor on which the substance is disposed, the first light source, and the second light source. A mask having a light-transmitting part through which light is transmitted and a light-shielding part that blocks the light; and a processing circuit, wherein the image sensor is (a) when illuminated by the first light source, Acquiring a first image, (b) acquiring a second image of the substance when illuminated with the second light source, and (c) when illuminated with a light source included in the light source group Acquire a fourth image of the substance, Thus, the first image and the second image are acquired via the mask positioned between the image sensor and the first light source and the second light source, and the fourth image is Acquired without passing through the mask, and the processing circuit selects an image having a large luminance value or an image having a small luminance value from the first image and the second image, and the selected image and A fifth image of the substance is generated by deriving a difference based on the fourth image.

  Note that these comprehensive or specific aspects may be realized by a system, method, integrated circuit, computer program, or computer-readable recording medium, and an apparatus, system, method, integrated circuit, computer program, and recording medium. It may be realized by any combination of the above. The computer-readable recording medium includes a non-volatile recording medium such as a CD-ROM (Compact Disc-Read Only Memory).

  According to the present disclosure, the image generation apparatus can be reduced in size. Additional benefits and advantages of one aspect of the present disclosure will become apparent from the specification and drawings. This benefit and / or advantage may be provided individually by the various aspects and features disclosed in the specification and drawings, and not all are required to obtain one or more thereof.

2 is a perspective view of a cell culture container according to Embodiment 1. FIG. 3 is a cross-sectional view of the cell culture container according to Embodiment 1. FIG. 2 is a block diagram illustrating an example of a functional configuration of the image generation apparatus according to Embodiment 1. FIG. It is a figure which shows typically an example of the structure of the illuminator which concerns on Embodiment 1. FIG. 5 is a diagram schematically showing an example of an arrangement of a plurality of point light sources according to Embodiment 1. FIG. It is a figure which shows typically an example of the structure of the mask which concerns on Embodiment 1. FIG. 3 is a schematic diagram illustrating a positional relationship between the image sensor and the object according to Embodiment 1. FIG. 6 is a diagram illustrating an example of a relationship between a position of a point light source and light received by an image sensor 150 according to Embodiment 1. FIG. 6 is a diagram illustrating an example of content stored in a storage unit according to Embodiment 1. FIG. 3 is a block diagram illustrating an example of a functional configuration of a light / dark image processing unit according to Embodiment 1. FIG. 4 is a flowchart illustrating an example of an operation of the image generation apparatus according to the first embodiment. 3 is a flowchart illustrating an example of an operation of an imaging unit according to the first embodiment. FIG. 6 is a schematic diagram for explaining a specific example of a cycle of a bright section and a dark section according to the first embodiment. FIG. 6 is a schematic diagram for explaining a specific example of a cycle of a bright section and a dark section according to the first embodiment. FIG. 4 is a diagram illustrating light intensity at each position on the image sensor according to the first embodiment. 4 is a flowchart illustrating an example of an operation of a light and dark image processing unit according to the first embodiment. It is a figure which shows an example of the imaging | photography result by the image sensor using the mask which concerns on Embodiment 1. FIG. 6 is a flowchart illustrating an example of an operation of a bright and dark image processing unit according to a modification of the first embodiment. 6 is a block diagram illustrating an example of a functional configuration of an image generation device according to Embodiment 2. FIG. It is a figure which shows typically an example of the structure of the illuminator which concerns on Embodiment 2. FIG. 6 is a diagram illustrating an example of content stored in a storage unit according to Embodiment 2. FIG. 6 is a flowchart illustrating an example of an operation of an imaging unit according to the second embodiment. 10 is a flowchart illustrating an example of an operation of a light and dark image processing unit according to the second embodiment. 10 is a flowchart illustrating another example of the operation of the bright and dark image processing unit according to the second embodiment. 10 is a flowchart illustrating an example of an operation of an imaging unit according to a modification example of the second embodiment. 10 is a flowchart illustrating an example of an operation of a light and dark image processing unit according to a modification of the second embodiment. 12 is a flowchart illustrating another example of the operation of the bright and dark image processing unit according to a modification of the second embodiment. It is a figure which shows the image image | photographed in the state which irradiated the illumination by a point light source and was without a mask. 3 is a diagram showing an image shot and synthesized by the method of Embodiment 1. FIG. It is a figure which shows the brightness | luminance profile of the image of FIG. 26A, and the image of FIG. 26B. It is a figure which shows the image image | photographed in the state which irradiated the illumination by a point light source and was without a mask. It is a figure which shows the image image | photographed and synthesize | combined by the method of Embodiment 2. FIG. It is a figure which shows the brightness | luminance profile of the image of FIG. 28A, and the image of FIG. 28B.

  An image generation apparatus according to an aspect of the present disclosure is an image generation apparatus that generates an image of a light-transmitting substance, and includes a first light source that illuminates the substance, and a predetermined distance from the first light source. A second light source that illuminates the substance from a position that is far away, an image sensor in which the substance is disposed, a translucent part through which light from the first light source and the second light source passes, and the light The image sensor includes a light shielding portion for shielding, a mask positioned between the image sensor and the first light source and the second light source, and a processing circuit. The image sensor is illuminated by the first light source. A first image of the substance is obtained when illuminated, and a second image of the substance is obtained when illuminated by the second light source, and the processing circuit applies the first image to the first image. The luminance value of the included pixel and the second image A Murrell pixel, by deriving a difference between the luminance values of the pixels at the same positions as the pixels included in the first image to generate a third image of said material.

  Thereby, the 1st image and 2nd image from which a light-and-dark pattern differs can be acquired by the illumination by the 1st light source and 2nd light source which are in the mutually distant position. Accordingly, by deriving the difference between them, noise caused by scattered light or refracted light can be reduced, and a clear third image can be generated by direct light. Moreover, since it is only necessary to switch the light source that illuminates the substance in order to acquire two images with different light and dark patterns, there is no need to change the position of a structure such as a light source or a digital micromirror device, and the entire apparatus can be made compact Can be

  That is, the image generation apparatus according to an aspect of the present disclosure is a lensless microscope, and allows diffused light from a point light source to pass through, for example, a mask having a slit or a checker pattern. Then, based on an image taken under illumination with a first light / dark pattern and an image taken under illumination with a second light / dark pattern in which the first light / dark pattern is mixed, Generate an image of the object.

  As a result, a photographed image with reduced noise can be generated by photographing in two states of light and dark by multi-light source photographing without using a digital micromirror device. In other words, in a lensless imaging system where a substance is placed on an image sensor and captured by transmitted light, a high-quality image with reduced noise is generated by using an image captured in two states of light and dark. Can do.

  In addition, in a part of the light receiving surface of the image sensor, the first image is formed in a state where the translucent portion of the mask is disposed between the first light source that illuminates the substance and the region. And the second image may be acquired in a state where the light shielding portion of the mask is disposed between the region and the second light source that illuminates the substance. For example, a partial area of the light receiving surface of the image sensor may be obtained when the plurality of light sources including the first light source and the second light source are sequentially illuminating the substance. And a plurality of images having different brightness values including the second image, wherein the processing circuit is an image having a maximum brightness value among the plurality of images, and the first light source is the An image acquired by the image sensor when the substance is illuminated is selected as the first image, and the second light source is an image having a minimum luminance value among the plurality of images. Selects an image acquired by the image sensor when illuminating the substance as the second image.

  As a result, the difference between the first image captured in the bright section and the second image captured in the dark section can be derived, noise is further reduced, and a clearer third image is generated. can do.

  The first light source is located half a cycle away from the second light source, and a partial region of the light receiving surface of the image sensor includes a plurality of light sources including the first light source and the second light source. When each sequentially illuminates the material, an image having a maximum luminance value and an image having a minimum luminance value are obtained, and the processing circuit includes a first luminance maximum image having a maximum luminance value. Of the image group and the second image group including the minimum luminance image having the minimum luminance value, an image group having a small luminance value variance is selected. (I) The selected image group is the first image group. In the case of an image group, the image has the maximum luminance value in the first image group, and is acquired by the image sensor when the first light source is illuminating the substance. An image is selected as the first image, and the second An image acquired by the image sensor when the light source illuminates the substance is selected as the second image, and (ii) the selected image group is the second image group In the second image group, an image having a minimum luminance value, the image acquired by the image sensor when the second light source illuminates the substance, The image acquired by the image sensor when the first light source is illuminating the substance may be selected as the first image.

  Thereby, even if the maximum value or the minimum value of the brightness is unstable, the first image captured in the bright section and the second image captured in the dark section are appropriately selected based on the period. can do. Thereby, a clearer image can be generated.

  An image generation apparatus according to an aspect of the present disclosure is an image generation apparatus that generates an image of a light-transmitting substance, and includes a first light source that illuminates the substance, and a predetermined amount from the first light source. A light source group including a plurality of light sources including a second light source that illuminates the substance from a position separated by a distance from the image sensor, an image sensor on which the substance is disposed, the first light source, and the second light source. A mask having a light-transmitting part through which light is transmitted and a light-shielding part that blocks the light; and a processing circuit, wherein the image sensor is (a) when illuminated by the first light source, Acquiring a first image, (b) acquiring a second image of the substance when illuminated with the second light source, and (c) when illuminated with a light source included in the light source group Acquire a fourth image of the substance, Thus, the first image and the second image are acquired via the mask positioned between the image sensor and the first light source and the second light source, and the fourth image is Acquired without passing through the mask, and the processing circuit selects an image having a large luminance value or an image having a small luminance value from the first image and the second image, and the selected image and A fifth image of the substance is generated by deriving a difference based on the fourth image. For example, each of the first image, the second image, the fourth image, and the fifth image has a luminance value corresponding to the same pixel included in the image sensor.

  Thereby, the illumination by the first light source or the second light source makes it possible to acquire the first image or the second image having a light and dark pattern and the fourth image that is uniformly bright and has no light and dark pattern. Therefore, by deriving a difference based on them, noise caused by scattered light or refracted light can be reduced, and a clear fifth image can be generated by direct light. In addition, in order to obtain an image having a light and dark pattern and an image that is uniformly bright and has no light and dark pattern, it is not necessary to change the position of a complicated structure such as a digital micromirror device. can do.

  Further, the area ratio of the light transmitting portion and the light shielding portion in the mask may be 1: 1. At this time, the processing circuit generates a luminance value of the fifth image by deriving a difference between twice the luminance value of the selected image and the luminance value of the fourth image. Also good. For example, if the processing circuit selects an image having a large luminance value from the first image and the second image, the processing circuit starts from the twice the luminance value of the selected image. The brightness value of the fifth image is generated by subtracting the brightness value of the image. Alternatively, when the processing circuit selects an image having a small luminance value from the first image and the second image, the luminance of the selected image is determined from the luminance value of the fourth image. The brightness value of the fifth image is generated by subtracting twice the value.

  Thus, by using the area ratio between the light transmitting portion and the light shielding portion of the mask, noise due to scattered light or refracted light can be reduced, and a clear fifth image can be generated by direct light.

  These general or specific aspects may be realized by a recording medium such as an apparatus, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM. The apparatus, the method, the integrated circuit, and the computer program Alternatively, it may be realized by any combination of recording media.

  Hereinafter, an image generation device and an image generation method according to an aspect of the present disclosure will be specifically described with reference to the drawings.

  It should be noted that each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the scope of the claims. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment 1)
The image generation apparatus according to Embodiment 1 includes an illuminator including a plurality of point light sources and a mask having a light-transmitting portion that transmits light and a light-blocking portion that blocks light, such as a slit or a checker pattern. And an image sensor and a processing circuit. A plurality of point light sources having different positions in the illuminator sequentially illuminate an object located on the image sensor. At this time, the image sensor shoots the object while the light and dark patterns of light reaching the image sensor and the object are switched. As a result, a plurality of images having different brightness patterns are acquired. For each pixel of the image sensor, the processing circuit obtains a luminance difference between the image having the highest luminance of the pixel and the image having the lowest luminance among the plurality of images, and calculates the luminance difference of each pixel. An image having a luminance value is generated.

  Here, the image generation apparatus according to Embodiment 1 images, for example, a cell in a mixed solution contained in a cell culture container as an object. First, the cell culture container and the like will be described in detail. Note that the object to be imaged in the present embodiment is a cell, for example, but may be a substance having translucency, and may be a substance other than a cell.

[1. Structure of cell culture container]
1A is a perspective view of a dish-type cell culture container 1000 according to Embodiment 1. FIG. FIG. 1B is a cross-sectional view of the dish-type cell culture vessel 1000 according to Embodiment 1. As shown in FIGS. 1A and 1B, the cell culture container 1000 includes a container part 1010, an illuminator 140, and an image sensor 150. The cell culture container 1000 is a dish-type container called a petri dish or a petri dish, but may be a horizontally placed flask-type container.

  The container unit 1010 is a container that contains a mixed solution containing cells and a culture solution. That is, the container part 1010 is a container in which a liquid mixture is located inside. The container part 1010 is a transparent container made of glass or resin, for example, and includes a lid part 1011 and a main body part 1012.

  The main body 1012 is a bottomed cylindrical member that forms the bottom and sides of the container 1010.

  The lid portion 1011 is a bottomed cylindrical member that closes the opening of the main body portion 1012 by being fitted to the main body portion 1012. The lid portion 1011 forms the upper portion of the container portion 1010.

  The illuminator 140 is provided on the inner surface of the lid portion 1011 and irradiates the mixed liquid in the container portion 1010 with light. As a result, the irradiated light is transmitted through the mixed solution and output as transmitted light. That is, the transmitted light is light that has passed through the liquid mixture from the illuminator 140, and is light that has been refracted and attenuated by the liquid mixture that is a translucent substance. Specifically, the illuminator 140 is fixed to the inner surface of the lid portion 1011, and irradiates the mixed liquid in the container portion 1010 with light from above. The illuminator 140 may be fixed to the outer surface of the lid portion 1011.

  Further, in the present embodiment, the illuminator 140 protrudes from the upper part of the container part 1010 to the inside of the container part 1010, and the light emission surface 140 s in the illuminator 140 is formed by the cells C 1 and culture in the container part 1010. Located in the liquid mixture containing the liquid L1. That is, the light exit surface 140 s of the illuminator 140 is positioned below the liquid level L <b> 2 of the mixed liquid and above the bottom of the container portion 1010.

  The image sensor 150 is provided at the bottom of the container 1010 and receives transmitted light output from the mixed solution. The image sensor 150 is a solid-state image sensor such as a CCD image sensor (Charge Coupled Device Image Sensor) or a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor). In the image sensor 150, a plurality of pixels are arranged in a matrix. The light emitted from the illuminator 140 is incident on each pixel of the image sensor 150. The image sensor 150 captures an optical image of a cell formed on the light receiving surface of the image sensor 150 when irradiated with light.

  Specifically, as shown in FIG. 1B, the image sensor 150 is fitted into an opening formed at the bottom of the container portion 1010. The light receiving surface of the image sensor 150 covered with the transparent protective film 150 a is exposed to the space inside the container portion 1010. The cell C1 is submerged in the culture solution L1 filled in the container portion 1010, and is cultured in a state of being in direct contact with the transparent protective film 150a. As shown in FIG. 1B, no condensing lens exists between the cell C1 and the image sensor 150.

  Thus, the cell C1, which is the object to be imaged, is cultured in a state of being placed on the light receiving surface of the image sensor 150 via the transparent protective film 150a.

[2. Configuration of Image Generation Device]
FIG. 2 is a functional block diagram of the image generation apparatus 10 according to the first embodiment. An image generation apparatus 10 illustrated in FIG. 1 is an apparatus that generates an image of a light-transmitting substance such as a cell, and includes an imaging unit 100, a storage unit 110, a light / dark image processing unit 120, and an output unit 130. With.

[2-1. Shooting part]
First, the configuration of the imaging unit 100 will be described. The imaging unit 100 includes an illuminator 140, an image sensor 150, and a control unit 160. The imaging unit 100 acquires a captured image (photographic image) of a substance that is an object. Here, the photographing unit 100 does not have a focus lens. The image generation apparatus 10 according to the first embodiment includes the control unit 160, the storage unit 110, and the output unit 130. However, these components are not essential, and the image generation device 10 includes these components. It does not have to be provided.

  The illuminator 140 according to Embodiment 1 includes a first light source 141A, a second light source 141B, and a mask 142. The first light source 141A is a point light source that illuminates a substance. The second light source 141B is a point light source that illuminates the substance from a position away from the first light source 141A by a predetermined distance. The mask 142 includes a light-transmitting part through which light from the first light source 141A and the second light source 141B is transmitted and a light-shielding part that blocks the light. The image sensor 150, the first light source 141A, and the second light source are masked. It is located between the light source 141B. In addition, the above-mentioned predetermined distance is a distance of 1/3 of a period described later.

  Hereinafter, the details of the illuminator 140 will be described.

  FIG. 3 is a diagram schematically illustrating an example of the structure of the illuminator 140 in the first embodiment. Specifically, the illuminator 140 includes a plurality of point light sources 141 including the first light source 141A and the second light source 141B described above, and a mask 142. The mask 142 has a slit or a checker pattern. That is, the mask 142 has a light transmitting part that transmits light and a light shielding part that blocks light.

  FIG. 4 is a schematic diagram illustrating an example of the arrangement of the plurality of point light sources 141. In the example of FIG. 4, the plurality of point light sources 141 are arranged at equal intervals. Each of the plurality of point light sources 141 has, for example, a light emitting surface with a diameter of 10 μm that emits light, and is attached to the light shielding plate so that the light emitting surface is exposed from a pinhole with a diameter of 10 μm provided on the light shielding plate. Yes. It is assumed that light spreads uniformly from the point light source 141 in all directions.

  FIG. 5 shows an example of the mask 142. Specifically, FIG. 5A shows an example of a mask 142 having a slit, and FIG. 5B shows an example of a mask 142 having a checker pattern. In addition, the black part in FIG. 5 is the above-mentioned light-shielding part, and the white part is the above-mentioned translucent part. In the example of FIG. 5, in the slit of (a), the width of the light-shielding part of the black line and the light-transmitting part of the white line are the same. In the checker pattern of (b), the black square light-shielding portion and the white square light-transmitting portion have the same size. Further, the plurality of light transmitting portions and the plurality of light shielding portions are set at equal intervals on the mask 142. That is, the plurality of light transmitting portions and the plurality of light shielding portions of the mask 142 are periodically and regularly arranged. Further, the width of the line in the slit or the length of one side of the square in the checker pattern is set so that a part of the object is brightened and the remaining part is darkened by the light flux from any point light source 141 through the mask 142. Has been. That is, the width or the length of one side of the square is such that the object by the luminous flux and the light / dark pattern of light on the image sensor 150 divide the object into at least two regions (bright region and dark region). Is set. For example, when the object is 100 μm, the width of the line in the slit or the length of one side of the square of the checker pattern is, for example, 30 μm. The mask 142 is realized, for example, by depositing a metal on glass.

  FIG. 6 is an example of an object installed on the image sensor 150. In FIG. 6, the transparent protective film 150a and the like are omitted. An object to be photographed is installed directly on the image sensor 150. The object is, for example, a plurality of translucent substances. A plurality of substances are positioned three-dimensionally overlapping. Specific examples of the substance are cells or cultured cells. In the example of FIG. 6, the object is an early embryo.

  The image sensor 150 has a plurality of pixels, and the above-described substances are arranged thereon. Each pixel of the image sensor 150 is arranged on the light receiving surface, and acquires the intensity (that is, the luminance value) of light emitted from the plurality of point light sources 141. The image sensor 150 acquires a captured image based on the light intensity acquired by each pixel. That is, the image sensor 150 acquires a first image of a substance when illuminated by the first light source 141A among the plurality of point light sources 141, and uses the second light source among the plurality of point light sources 141. When illuminated, a second image of the material is acquired.

  Examples of the image sensor 150 are a CMOS (Complementary Metal-Oxide Semiconductor) image sensor or a CCD (Complementary Metal-Oxide Semiconductor) image sensor.

  The plurality of point light sources 141 of the illuminator 140 emit light in order. The plurality of point light sources 141 are arranged at different positions and irradiate the object through the mask 142 from different directions.

  The control unit 160 controls light irradiation by the plurality of point light sources 141 and photographing by the image sensor 150. Specifically, the control unit 160 controls the order in which the plurality of point light sources 141 emit light and the time interval at which the plurality of point light sources 141 emit light. The control unit 160 includes a computer system (not shown) including a CPU (Complementary Metal-Oxide Semiconductor), a RAM (Random Access Memory), a ROM (Read-Only Memory), and the like. Some or all of the functions of the components of the control unit 160 may be achieved by the CPU executing a program recorded in the ROM using the RAM as a working memory. In addition, some or all of the functions of the components of the control unit 160 may be achieved by a dedicated hardware circuit.

  FIG. 7 shows an example of the relationship between the position of the point light source 141 and the light received by the image sensor 150. For example, as shown in FIG. 7A, the light transmitted from the point light source 141 through the light transmitting portion of the mask 142 illuminates a part of the substance and a region b of the light receiving surface of the image sensor 150. . At this time, the light that illuminates a part of the substance and the region b of the image sensor 150 includes not only direct light from the point light source 141 but also refracted light. The refracted light is light that is transmitted from the point light source 141 through the light transmitting portion of the mask 142 and is refracted, reflected, or scattered by the culture medium or a transmissive material. On the other hand, a part of the direct light from the point light source 141 is blocked by the light shielding portion of the mask 142 and does not reach the region a of the light receiving surface of the image sensor 150. However, the refracted light reaches the region a. Accordingly, the region a of the light receiving surface of the image sensor 150 does not receive direct light and is dark because it receives refracted light, and the region b is bright because it receives direct light and refracted light.

  However, when the point light source 141 shown in FIG. 7A is turned off and the other point light sources 141 shown in FIG. 7B are turned on, the region e of the light receiving surface of the image sensor 150 is different from the region a. It becomes dark because it receives only refracted light. In addition, a region f different from the region b on the light receiving surface of the image sensor 150 is brightened because it receives refracted light and direct light.

  That is, when the position of the point light source 141 that illuminates the substance is different, a part of the light receiving surface of the image sensor 150 becomes brighter to receive refracted light and direct light, or does not receive direct light. It becomes dark to receive refracted light. In the first embodiment, a plurality of point light sources 141 arranged as shown in FIG. 4, specifically, a plurality of points along a direction in which light-transmitting portions and light-shielding portions of the mask 142 are alternately arranged regularly. The point light source 141 illuminates in order. Therefore, a part of the light receiving surface of the image sensor 150 (specifically, a pixel) periodically becomes brighter or darker depending on the position of illumination. Hereinafter, the section of the illumination position that becomes bright is referred to as a bright section, and the section of the illumination position that becomes dark is referred to as a dark section. In the first embodiment, for each region (for example, pixel) of the light receiving surface of the image sensor 150, the change in the brightness (that is, the luminance value) depending on the illumination position is used, so that the material from which the noise of the refracted light is removed. Generate an image.

[2-2. Storage unit]
The storage unit 110 stores the image acquired by the image sensor 150 in association with the position of the point light source 141 that is illuminated or turned on at the time of shooting set by the control unit 160 (hereinafter also referred to as an illumination position).

  FIG. 8 is an example of information stored in the storage unit 110. The storage unit 110 stores the ID for identifying the image acquired by the image sensor 150 in association with the illumination position of the point light source 141 that was turned on at the time of shooting. The illumination position is indicated, for example, as a point on the coordinate with the upper left corner of the plane composed of a plurality of effective pixels of the image sensor 150 as the origin, the horizontal direction of the image sensor 150 as the x axis, and the vertical direction as the y axis. In the example of the illuminator 140 of the first embodiment, the plurality of point light sources 141 are arranged on a plane parallel to the surface (that is, the light receiving surface) of the image sensor 150 so as to face the surface of the image sensor 150. Yes. That is, the distance from the surface of the image sensor 150 to each of all the point light sources 141 is the same. Therefore, the illumination position is expressed in two dimensions. The illumination position may be expressed in three dimensions. Further, when all the point light sources 141 are arranged linearly, the illumination position may be expressed in one dimension.

[2-3. Bright and dark image processing unit]
The bright / dark image processing unit 120 is realized by at least one control circuit or processing circuit. In the first embodiment, the light / dark image processing unit 120 generates the third image of the substance by deriving the difference between the first image and the second image. Here, in the first embodiment, each of the first image, the second image, and the third image has a luminance value corresponding to the same pixel included in the image sensor 150.

  FIG. 9 is a functional block diagram showing a detailed configuration of the light / dark image processing unit 120. As illustrated in FIG. 9, the light / dark image processing unit 120 includes a data acquisition unit 121, a maximum value determination unit 122, a minimum value determination unit 123, a calculation unit 124, an image generation unit 125, and a pixel selection unit 126. And a storage unit 127.

  The data acquisition unit 121 acquires an image used for image processing, that is, a luminance value for each pixel included in the image and an illumination position corresponding to the image from the storage unit 110.

  The pixel selection unit 126 selects a pixel for performing luminance calculation from a plurality of pixels of the image sensor 150, that is, a plurality of pixels in an image to be generated. Hereinafter, the pixel selected by the pixel selection unit 126 is also referred to as a selected pixel.

  The maximum value determination unit 122 compares the luminance values of the pixels at the same position as the pixel selected by the pixel selection unit 126 of each of the plurality of images stored in the storage unit 110, and determines the pixel having the maximum luminance value. An image to be included is specified from the plurality of images. The maximum value determination unit 122 determines the maximum luminance value as the maximum value of the luminance of the selected pixel.

  The minimum value determination unit 123 compares the luminance values of the pixels at the same position as the pixel selected by the pixel selection unit 126 in each of the plurality of images stored in the storage unit 110, and determines the pixel having the minimum luminance value. An image to be included is specified from the plurality of images. The minimum value determination unit 123 determines the minimum luminance value as the minimum value of the luminance of the selected pixel.

  The storage unit 127 stores the image and illumination position acquired by the data acquisition unit 121 from the storage unit 110, the maximum value determined by the maximum value determination unit 122, and the minimum value determined by the minimum value determination unit 123.

  For each selected pixel of the image to be generated, the calculation unit 124 calculates the selected pixel determined by the minimum value determining unit 123 from the maximum value determined by the maximum value determining unit 122 and corresponding to the selected pixel. Decrease the corresponding minimum value. Thereby, the luminance value of each pixel of the image to be generated is calculated.

  The image generation unit 125 generates an image including the luminance value of each pixel calculated by the calculation unit 124.

  That is, in the first embodiment, the partial area of the light receiving surface of the image sensor 150 is the first when the plurality of light sources including the first light source and the second light source sequentially illuminate the substance. A plurality of images having different brightness values are acquired, including the first image and the second image. Specifically, a part of the light receiving surface of the image sensor 150 is a pixel. In this case, the light / dark image processing unit 120 serving as a processing circuit is an image having the maximum luminance value among the plurality of images, and is used by the image sensor 150 when the first light source 141A is illuminating the substance. The acquired image is selected as the first image. In addition, the light / dark image processing unit 120 is an image having the minimum luminance value among the plurality of images, and the image acquired by the image sensor 150 when the second light source 141B is illuminating the substance, Select as second image.

  The data acquisition unit 121, the maximum value determination unit 122, the minimum value determination unit 123, the calculation unit 124, and the image generation unit 125 included in the light / dark image processing unit 120 are a CPU (Complementary Metal-Oxide Semiconductor), a RAM (Random). It is configured by a computer system (not shown) including an access memory (ROM), a ROM (read-only memory), and the like. Some or all of the functions of the components of the light / dark image processing unit 120 may be achieved by the CPU executing a program recorded in the ROM using the RAM as a working memory. Further, some or all of the functions of the components of the light / dark image processing unit 120 may be achieved by a dedicated hardware circuit.

[2-4. Output section]
The output unit 130 is an output device that presents an image generated by the light and dark image processing unit 120, or a unit that outputs the image as electronic data. An output device that presents an image is, for example, a display. The means for outputting as electronic data is, for example, a USB connector.

[3. Operation of image generation apparatus]
Next, the operation of the image generation apparatus 10 configured as described above will be described.

  FIG. 10 is a flowchart illustrating an example of the operation of the image generation apparatus 10 according to the first embodiment.

  The image generation apparatus 10 starts an operation in response to an operation start command (not shown).

(Step S1200)
The imaging unit 100 performs multi-light source imaging. That is, the imaging unit 100 uses each of the plurality of point light sources 141 of the illuminator 140 in order to illuminate the object with light that has passed through the mask 142, and images a plurality of images of the object. Specifically, the imaging unit 100 records the intensity of light reaching the light receiving surface of the image sensor 150 each time the plurality of point light sources 141 of the illuminator 140 illuminate the object, thereby the object. Get the image. The acquired image is stored in the storage unit 110 together with position information (that is, an illumination position) of the point light source 141 that has illuminated the object at the time of shooting. Here, the positions of the plurality of point light sources 141 are fixed with respect to the image sensor 150, and the position information of each of the plurality of point light sources 141 is predetermined. Details of the multiple light source photographing will be described later.

(Step S1300)
The light / dark image processing unit 120 performs light / dark image processing on the light and dark portions on the image captured in step S1200 by the mask 142 of the illuminator 140. In step S1200, illumination is performed with each of the point light sources 141 at a plurality of positions different from each other, so that the positions of the bright portion and the dark portion are different for each image. The bright and dark image processing unit 120 compares the luminance of the same pixel position between the images for the plurality of images taken in step S1200. The bright / dark image processing unit 120 determines the luminance value for each pixel by subtracting the minimum value from the maximum luminance value for each pixel. Details of the light / dark image processing will be described later.

(Step S1400)
The image generation unit 125 of the bright / dark image processing unit 120 generates and outputs an image based on the luminance value determined for each pixel in step S1300. Thereafter, the image generating apparatus 10 ends the operation.

  As described above, the image generation method according to Embodiment 1 is an image generation method for generating an image of a light-transmitting substance, and includes the first light source 141A, the second light source 141B, and the image sensor 150. And a mask 142 are used. In this image generation method, in step S1200, (a) when illuminated by the first light source 141A, the image sensor 150 acquires a first image of the substance, and (b) by the second light source 141B. When illuminated, the image sensor 150 acquires a second image of the material. Further, in this image generation method, in step S1300 and step S1400, (c) the third image of the substance is generated by deriving the difference between the luminance value of the first image and the luminance value of the second image. To do.

[3-1. Multi-light source shooting]
Here, details of the operation of the photographing unit 100 in step S1200 illustrated in FIG. 10 will be described.

  FIG. 11 is a flowchart illustrating an example of the operation of the imaging unit 100.

(Step S1210)
The control unit 160 refers to a list indicating a plurality of predetermined illumination positions, that is, a list indicating the positions of the plurality of point light sources 141 of the illuminator 140 (hereinafter referred to as an illumination position list), and performs illumination from each illumination position. It is determined whether or not shooting of the target object has been completed.

  Here, when photographing with illumination from all illumination positions included in the illumination position list has been completed (yes in step S1210), the process proceeds to step S1300. On the other hand, if photographing with illumination from any illumination position in the illumination position list has not been completed (No in step S1210), the process proceeds to step S1120. In addition, the case where the imaging | photography with the illumination from all the illumination positions included in the illumination position list | wrist is meaning that the imaging | photography of the image for 2 periods is complete | finished. Here, in each pixel, one period is a section obtained by combining a bright section in which direct light and refracted light are received and a dark section in which direct light is not received and refracted light is received. An image for two cycles is an illumination position that receives direct light and refracted light when a plurality of images photographed by illumination from a plurality of illumination positions are arranged in the order of arrangement of illumination positions in each pixel. Photographed by a plurality of lights arranged in the range of illumination positions, each including a bright section that is a range and a dark section that is a range of an illumination position that does not receive direct light and receives refracted light twice Means multiple images. Details of the two cycles will be described later with reference to FIGS. 12A and 12B.

(Step S1220)
The control unit 160 selects an illumination position that has not yet been illuminated from among a plurality of illumination positions included in the illumination position list, and outputs a control signal to the illuminator 140. This control signal includes the selected illumination position. Each illumination position in the illumination position list, that is, the position of each point light source 141 is indicated by a number assigned to each point light source 141, for example. Or each illumination position is shown by the coordinate value in xyz space which makes the surface of the image sensor 150 xy plane, or the coordinate value in the xy plane set on the surface parallel to the image sensor 150 surface, for example. The selection of the illumination position is performed, for example, in ascending order of the illumination position list.

(Step S1230)
The illuminator 140 starts illumination of the object according to the control signal output from the control unit 160 in step S1220. That is, among the plurality of point light sources 141 included in the illuminator 140, the point light source 141 at the illumination position selected in step S1220 starts light irradiation.

(Step S1240)
While the object is illuminated by the point light source 141, the image sensor 150 acquires an image formed by the light transmitted through the object from the point light source 141 through the mask 142.

(Step S1250)
The control unit 160 associates the image acquired in step S1240 with the illumination position of the point light source 141 illuminated at the time of acquisition of the image and outputs the associated image to the storage unit 110. The storage unit 110 stores the image and the illumination position in association with each other.

(Step S1260)
Then, the control part 160 outputs a control signal to the illuminator 140, and stops the illumination to a target object. Note that the stop of the illumination may not be performed in accordance with a control signal from the control unit 160. For example, the illuminator 140 measures the time length from the start of illumination by one of the point light sources 141, and activates the illumination when the time length exceeds a predetermined time length. You may stop at any time. Alternatively, after the image sensor 150 finishes acquiring an image in step S1240, the image sensor 150 may output a control signal for stopping the illumination to the illuminator 140. After step S1260, the process returns to step S1210.

  By repeating the processing from step S1210 to step S1260, the object is sequentially irradiated with light from the point light sources 141 at all the illumination positions included in the illumination position list. Each time the object is irradiated with light through the mask 142, the image sensor 150 acquires an image.

[3-2. Point light source arrangement]
Here, the illumination position, that is, the positions of the plurality of point light sources 141, the bright section, the dark section, and the cycle including the bright section and the dark section will be described in detail.

  FIG. 12A is a schematic diagram for explaining light and dark patterns depending on the position of the point light source 141 and the position of the mask 142 in an arbitrary pixel on the image sensor 150. The black-painted rectangle of the mask 142 indicates a light-shielding portion, and the white rectangle indicates a light-transmitting portion that transmits light. Assume that a point light source 141a, a point light source 141b, and a point light source 141c exist.

  The light reaching the arbitrary pixel A on the image sensor 150 from the point light source 141a is the light ray 1 and passes on the boundary line between the light shielding portion 142b and the light transmitting portion 142a of the mask 142. The light reaching the pixel A from the point light source 141b is the light ray 2 and passes on the boundary line between the light shielding portion 142b and the light transmitting portion 142c of the mask 142. When the point light source 141 is positioned on a straight line including the point light source 141a and the point light source 141b, on the left side from the point light source 141a in FIG. 12A toward the paper surface, and on the right side from the point light source 141b toward the paper surface (FIG. 12A, when the point light source 141 is positioned in a section indicated as “dark section in the pixel A”, the light beam emitted from the point light source 141 to the pixel A is blocked by the light shielding portion 142b of the mask 142. That is, with respect to the pixel A, the section on the straight line between the position of the point light source 141a and the point light source 141b (in FIG. 12A, the point light source 141 is located in the section indicated as "dark section at pixel A"). In this case, the direct light from the point light source is blocked by the light shielding portion 142b of the mask 142, and the light beam does not reach. Since the refracted light arrives but the direct light does not reach, the brightness value is relatively low and a dark image is acquired as an image (hereinafter referred to as a dark interval). The dark section is a continuous section on a straight line between the place where the point light source 141a is assumed to be present and the place where the point light source 141b is assumed to be present, but one point light source existing in the dark section is present. The amount of light detected by the pixel A when illuminated is smaller than the amount of light detected by the pixel A when one point light source existing in the light section described below is illuminated. A series of times when one or more light sources existing in one dark section are turned on may be called a dark section.

  On the other hand, the light beam reaching the pixel A from the point light source 141c is the light beam 3 and passes on the boundary line between the light transmitting portion 142c and the light shielding portion 142d of the mask 142. When the point light source 141 is positioned on a straight line including the point light source 141b and the point light source 141c, on the left side from the point light source 141b in FIG. 12A toward the paper surface, and on the right side from the point light source 141c toward the paper surface (FIG. 12A, when the point light source 141 is positioned in a section indicated as “bright section in the pixel A”, the light beam emitted from the point light source 141 to the pixel A is transmitted through the translucent portion 142c of the mask 142. The pixel A on the surface of the image sensor 150 is reached. That is, with respect to the pixel A, the section on the straight line between the position of the point light source 141b and the point light source 141c (in FIG. 12A, the point light source 141 is positioned in the section indicated as “bright section at pixel A”. ) Is a section in which direct light from a point light source arrives and refracted light arrives at the same time, and a bright image is acquired as a relatively high brightness value (hereinafter referred to as a bright section). . The bright section is a continuous section on a straight line between a place where the point light source 141b is assumed to be present and a place where the point light source 141c is assumed to be present, but one point light source existing in the bright section is present. The amount of light detected by the pixel A when illuminated is greater than the amount of light detected by the pixel A when one point light source existing in the dark section is illuminated. A series of periods in which one or a plurality of light sources existing in one bright section is lit may be referred to as a bright section.

  Next to one bright section, which is a time period during which one or more point light sources existing in a bright section (place) corresponding to one transmissive part of the mask 142 are lit, the mask 142 adjacent to the one transmissive part. One dark section, which is a section of the time during which the point light source existing in the dark section (location) corresponding to one light-shielding portion is turned on, is defined as one cycle of illumination. In other words, when each of the plurality of point light sources 141 in the illuminator 140 is arranged at equal intervals and sequentially illuminates at predetermined time intervals, “one bright section (location) and one single bright section ( A period during which each of the plurality of point light sources 141 arranged in one dark section (place) adjacent to (place) sequentially illuminates is one cycle.

  In the point light source 141, when the priority at the time of shooting differs depending on the position of the point light source, a plurality of illuminations existing in one dark section or bright section may not be illuminated in a series of time sections. For example, it is assumed that the point light source 141 is arranged for every 1/6 of one cycle and photographed. The point light source 141 is disposed at least every 1/3 of one cycle. That is, illumination every 1/3 of one cycle is absolutely necessary, the position of these point light sources is set to a high priority position, and the position of the point light sources arranged between 1/3 is low priority. Position. If the shooting time is not enough, shooting with a point light source at a higher priority position is performed first, and shooting with a point light source at a lower priority position ends with shooting at a higher priority position lighting. Done later. In such a case, the six point light sources arranged in the section of one cycle (location) are not illuminated in a series of time sections. Therefore, each of the plurality of point light sources 141 arranged in one bright section (place) and one dark section (place) adjacent to this one bright section (place), which is one cycle as time, is sequentially There is no “period of illumination”. When the point light sources 141 are arranged at equal intervals, a plurality of light sources (places) arranged in one bright section (place) and one dark section (place) adjacent to the one bright section (place) A section in which each of the point light sources 141 is disposed is defined as one section, and a plurality of images that are photographed in a state where each of the plurality of point light sources 141 is illuminated in one section are provided for one period. An image.

  Even if the light existing in one light section (place) and the dark section (place) adjacent to that light section (place) is not illuminated in a series of time sections, one light section (place) and its light A state illuminated by illumination of one dark section (place) adjacent to the section (place) is regarded as a place, that is, one spatial cycle.

  In the pixel B having a different position from the pixel A in FIG. 12A, the boundary position between the bright section and the dark section is different from the boundary position with respect to the pixel A. However, the size of the bright and dark intervals for pixel A and the size of the bright and dark intervals for pixel B are the same. This is because the size of the light shielding part and the light transmitting part of the mask 142, the distance between the mask 142 and the surface of the image sensor 150, and the distance between the point light source 141 and the surface of the image sensor 150 with respect to the pixel A and the pixel B. Is the same. That is, in any pixel on the image sensor 150, the light / dark cycle is the same, but the illumination position in the bright section and the illumination position in the dark section differ depending on the position of the pixel.

  In step S1300 illustrated in FIG. 10, the light and dark image processing unit 120 subtracts the minimum value from the maximum value of the luminance of the pixels at the same position in each of the plurality of images acquired by photographing for each pixel of the image sensor 150. The maximum value and the minimum value of the luminance correspond to the bright section and the dark section of each pixel on the image sensor 150, respectively.

  That is, in the first embodiment, a part of the light receiving surface of the image sensor 150 is the first light source that illuminates the substance and the translucent portion of the mask 142 is disposed between the first light source and the area. 1 image is acquired. Thereby, the 1st image in a bright section is acquired. In addition, in a partial region of the light receiving surface of the image sensor 150, a second image is acquired in a state where the light shielding portion of the mask 142 is disposed between the second light source that illuminates the substance and the region. Thereby, the second image in the dark section is acquired.

  Here, the imaging conditions necessary to specify the maximum and minimum luminance values of all the pixels of the captured image are all the pixels on the image sensor 150 and illumination of the point light source 141 located in the bright section. This means that both the shooting by, and the shooting by illumination of the point light source 141 located in the dark section are performed.

  In each of all the pixels on the image sensor 150, even if they are the same point light source, whether the point light source is located in the bright section or the dark section differs depending on the pixel. For example, as shown in FIG. 12B, the light directly from the point light source 141d is blocked by the light shielding portion 142b in the pixel A and becomes a dark section, but the light emitted from the point light source 141a is transmitted through the mask 142 in the pixel B. The light passes through the portion 142a and reaches the pixel B. That is, in the pixel B, the point light source 141d is in the bright section.

  In two different pixels on the image sensor 150, the bright and dark sections of illumination have the same section size but different positions. Depending on the distance between the two pixels, the positions of the bright and dark sections are shifted. For example, this is the case with pixel A and pixel B in FIG. 12A. A point light source 141a ′, which is an illumination position that passes through the boundary between the pixel B and the light shielding portion 142b of the mask 142, and the light transmitting portion 142a, and a point that is an illumination position that passes through the boundary between the pixel B, the light shielding portion 142b of the mask 142, and the light transmitting portion 142c. If the light source 141b ′ is assumed, the positions of the point light source 141a and the point light source 141a ′, the point light source 141b and the point light source 141b ′ are different, but the distance between the point light source 141a and the point light source 141b, that is, the size of the dark section in the pixel A The distance between the point light source 141a ′ and the point light source 141b ′, that is, the size of the dark section in the pixel B is the same. The same applies to the adjacent bright sections, and in the pixels A and B, the sizes of the dark section and the bright section are the same. That is, the illumination cycle with respect to the illumination position for each pixel is the same.

  Therefore, for pixel A and pixel B, at least one point light source 141 arranged in the bright section in each pixel shown in FIG. 12A and one point light source arranged in the dark section in each pixel shown in FIG. 12A. 141 is necessary to shoot when illumination is performed. That is, when the distance obtained by adding the distance of one dark section and the distance of one bright section is a one-cycle distance, the distance between adjacent point light sources needs to be equal to or less than a half-cycle distance. Actually, when the light from the plurality of point light sources 141 is sequentially irradiated, the light passes directly through the light transmitting portion of the mask 142 and reaches the surface of the image sensor 150 and the light shielding portion of the mask 142 directly. The boundary with the dark section where the light is blocked and does not reach the surface of the image sensor 150 is not clear.

  FIG. 13 shows the light intensity at each position on the image sensor 150. For example, as shown in FIG. 13, the luminance at the boundary portion changes gently. Therefore, even if the distance between the adjacent point light sources 141 is a half cycle distance, that is, when the distance obtained by adding the distance of one dark section and the distance of one bright section is a one cycle distance, one cycle. Even if two point light sources are provided at a distance of 5 mm, it may not be possible to obtain the imaging results in the bright and dark sections with all the pixels on the image sensor 150. The inventors have empirically arranged the point light sources 141 at a distance interval of 1/3 period when the distance obtained by adding the distance of one dark section and the distance of one light section is one period distance, that is, When three point light sources are arranged at a distance of one cycle and the point light source 141 is sequentially turned on to perform photographing, it is confirmed that photographing results in the bright and dark sections can be obtained with all pixels on the image sensor 150. doing. That is, the interval between the plurality of point light sources 141 in the illuminator 140 is equal to or less than a distance of 1/2 cycle, and preferably equal to or less than a distance of 1/3 cycle. The predetermined distance between the first light source 141 </ b> A and the second light source 141 </ b> B described above is the difference between the illumination position in the bright section for an arbitrary pixel on the image sensor 150 and the illumination position in the dark section for the pixel. Is the distance between. Specifically, when the interval between the plurality of point light sources 141 is a distance of 1/3 period, the predetermined distance between the first light source 141A and the second light source 141B described above is one dark section. When the distance obtained by adding the distance and the distance of one bright section is a distance of one cycle, the distance is 1/3 of the cycle.

  In the illumination position list referred to in step S1210, for example, the point light sources 141 are arranged at a distance interval of 1/3 period, that is, three point light sources are arranged at a distance of one period, and the point light source is crossed across the slit. 14 is a list showing 7-point illumination positions when 141 is arranged. Here, since the illumination has two periods and the point light sources 141 are arranged at a distance interval of 1/3 period, seven points of illumination are arranged in a section of two periods including both ends of the illumination period. The direction across the slit is a direction in which the light shielding portions and the light transmitting portions of the mask 142 are alternately arranged.

[3-3. Bright and dark image processing]
Details of the operation of the bright and dark image processing unit 120 in step S1300 shown in FIG. 10 will be described.

  FIG. 14 is a flowchart showing an example of the operation of the bright and dark image processing unit 120 according to the first embodiment.

(Step S1301)
The data acquisition unit 121 of the light / dark image processing unit 120 determines whether or not a plurality of images and illumination positions corresponding to the plurality of images are stored in the storage unit 110. Each of the plurality of images and the plurality of illumination positions is an image and an illumination position acquired when the plurality of point light sources 141 arranged in the two-cycle section sequentially illuminate in step S1200. If the data acquisition unit 121 determines that the data is stored (yes in step S1301), the data acquisition unit 121 continues the bright / dark image processing (proceeds to step S1310).

(Step S1310)
The data acquisition unit 121 acquires the plurality of images acquired in step S1200 and the illumination positions corresponding to each of the plurality of images from the storage unit 110.

(Step S1320)
The pixel selection unit 126 determines whether or not the luminance value calculation processing has been completed for all the pixels included in the generated image. More specifically, the calculation process means the process from step S1320 to step S1360.

  When the calculation process has been completed for all the pixels included in the generated image (yes in step S1320), the light / dark image processing unit 120 ends the light / dark image process (proceeds to step S1400).

  If the calculation process has not been completed for any pixel included in the generated image (NO in step S1320), the light / dark image processing unit 120 continues the light / dark image process (proceeds to step S1330).

  The image to be generated includes fewer pixels than the image having the smallest number of pixels among the plurality of images acquired in step S1200. That is, the number of pixels of the generated image is smaller than the number of pixels of any of the plurality of acquired images.

(Step S1330)
The pixel selection unit 126 selects one pixel from a plurality of pixels included in the generated image. One pixel selected here, that is, the selected pixel is a pixel that has not yet been subjected to calculation processing among a plurality of pixels included in the generated image. Note that the initial value of the pixel value (that is, the luminance value) of the image to be generated is zero.

(Step S1341)
The maximum value determination unit 122 compares the luminance values of the pixels at the same position as the selected pixel included in each of the plurality of images captured at step S1200 for the pixel selected at step S1330 (ie, the selected pixel). To do. As a result of the comparison, the maximum value determination unit 122 determines the maximum luminance value as the maximum luminance value of the selected pixel.

(Step S1342)
The minimum value determination unit 123 compares the luminance values of the pixels at the same position as the selected pixel included in each of the plurality of images captured at step S1200 for the pixel selected at step S1330 (ie, the selected pixel). To do. As a result of the comparison, the minimum value determination unit 123 determines the minimum luminance value as the minimum luminance value of the selected pixel.

  Note that either step S1341 or step S1342 may be performed first. It may also be performed in parallel.

(Step S1350)
The calculation unit 124 subtracts the minimum luminance value determined in step S1342 from the maximum luminance value determined in step S1341.

(Step S1360)
The image generation unit 125 stores the luminance value calculated in step S1350 as the luminance value of the selected pixel.

  The bright and dark image processing unit 120 can generate the luminance values of all the pixels of the image to be generated by repeating steps S1320 to S1360.

[4. effect]
As described above, according to the image generation apparatus 10 according to the first embodiment, the light from the plurality of point light sources 141 having different positions is projected through the mask 142 onto the object and the image sensor 150, and transmitted light is transmitted. Take a picture. The mask 142 has a repeated pattern of a light shielding portion and a light transmitting portion such as a slit or a checker pattern. For each pixel (that is, selected pixel) in the image to be generated from a plurality of images acquired by this shooting, the light is passed through the light-transmitting portion of the mask 142 and directly captured as the light. Identified images. Then, the luminance value of the pixel at the same position as the selected pixel included in the identified image is determined as the maximum luminance value. Similarly, an image photographed in a state where direct light from the point light source 141 is blocked by the light-shielding portion of the mask 142 is specified for each selected pixel from a plurality of images obtained by photographing. Then, the luminance value of the pixel at the same position as the selected pixel included in the specified image is determined as the minimum luminance value. The maximum value of luminance indicates luminance according to direct light and scattered light or refracted light from the object. The minimum value of luminance indicates luminance according to scattered light or refracted light by an object other than direct light. By subtracting the minimum value from the maximum value of luminance, the luminance value obtained by direct light can be obtained. Thereby, noise caused by scattered light or refracted light can be reduced, and a clear image can be generated by direct light.

  Here, Non-Patent Document 1 discloses a scattered light removal technique using high-frequency illumination.

  In high-frequency illumination, it is conceivable to change the type of light output from the light source and the method for generating the light / dark pattern with the light applied to the object.

  In Non-Patent Document 1, it is assumed that the distance between the object and the light source and the mask is sufficiently large. Therefore, an image of an object is taken by illuminating with parallel light or light close to parallel light using a light source.

  A method for generating a light / dark pattern with light applied to an object includes (i) a method using light having a light / dark pattern, and (ii) a method of installing a light shielding mask between the illumination and the object. There is. Further, in consideration of moving the position of the light / dark pattern of the irradiated light on the upper surface of the object, there are (iii) a method of changing the position of the light source and (iv) a method of changing the position of the mask.

  When imaging an image using parallel light or light close to parallel light as in Non-Patent Document 1, the object passes through the mask even if the position of the mask is fixed and the position of the light source is moved. There is no change in the light applied to the light source. That is, (iii) it is difficult to adopt a method of moving the position of the light / dark pattern of the irradiated light by changing the position of the light source.

  On the other hand, in the case of a microscope having a condition in which the distance between the light source and the mask and the object is small, it is difficult to easily obtain parallel light. Therefore, it is conceivable that diffused light is used as the type of light output from the light source.

  When diffused light is used, if the distance from the mask to the object is large, the light / dark pattern spreads and it is difficult to obtain the effect of high-frequency illumination.

  For example, when the mask is brought close to the object, (iv) the movement of the mask may affect a minute object such as a cell. Specifically, there is a possibility that a minute object moves or rotates as the mask moves. When an image of a minute object such as a cell is captured, the distance between the light source and the mask and the object is reduced. Therefore, when the distance between the mask and the object is small and a light source that irradiates diffused light must be used, the position of the light brightness / darkness pattern of the light is fixed by fixing the mask and moving the light source. By moving, high-frequency illumination can be effectively performed.

<Effect of Embodiment 1>
FIG. 26A and FIG. 26B are images obtained by photographing a state in which a plurality of glass beads having an average diameter of 0.04 mm are hardened with a resin. The plurality of spheres shown in FIGS. 26A and 26B correspond to glass beads.

  The image in FIG. 26A is an image that is captured with a point light source illuminated and without a mask. The image in FIG. 26B is an image that is captured and synthesized by the method of the first embodiment. The image of FIG. 26B has a larger contrast and clear outline than the image of FIG. 26A.

  FIG. 27 shows the image brightness profiles of FIG. 26A and FIG. 26B. In FIG. 27, the luminance profile of the image of FIG. 26A is shown by a solid line, and the luminance profile of the image photographed and synthesized by the method of Embodiment 1 of FIG. 26B is shown by a broken line. The horizontal axis of FIG. 27 is a value indicating the horizontal axis position of the image by the number of pixels, and the vertical axis is a luminance value. The luminance profile shown in FIG. 27 was calculated at the position indicated by the arrow in FIGS. 26A and 26B.

  In the region where the horizontal axis of FIG. 27 is 50 or more, the luminance gradient of the image (broken line) in FIG. 26B is larger than the luminance gradient (solid line) of the image of FIG. 26A. Therefore, as shown in FIG. 27, it can be confirmed that an image captured and synthesized by the method of Embodiment 1 has a higher contrast than an image captured without a mask.

(Modification of Embodiment 1)
Next, a modification of the first embodiment will be described. In the first embodiment, each pixel is photographed in a range of one cycle or more, and for each selected pixel, the maximum luminance value and the minimum luminance value of the pixel at the same position as the selected pixel in each of the plurality of images are obtained. decide. Then, the luminance value of the selected pixel is obtained by subtracting the minimum luminance value from the maximum luminance value. In this modification, a luminance value having a small variance, that is, a stable luminance value between the maximum luminance value and the minimum luminance value is adopted as the maximum value or the minimum value of the luminance of the selected pixel. If the adopted brightness value is the maximum value, the minimum value of the brightness of the selected pixel is determined using the illumination position at which the maximum value is obtained and the above-described cycle. On the other hand, if the adopted luminance value is the minimum value, the maximum value of the luminance of the selected pixel is determined using the illumination position when the minimum value is obtained and the above-described cycle.

  FIG. 15 shows an example of a photographing result by the image sensor 150 using the mask 142. Specifically, FIG. 15 shows a case where the light from the point light source 141 is passed through the slit mask 142 and received by the surface of the image sensor 150 installed in parallel with the mask 142. An example of the obtained image is shown. In this example, there is no object. As shown in FIG. 15, unevenness in intensity occurs due to interference of light that has passed through the mask 142, and the illumination position where the maximum value or minimum value can be obtained may become unstable. In such a case, the image showing the maximum luminance value may be different from the image with the strongest direct light in the bright section. Alternatively, an image showing the minimum luminance value may be different from an image with the least direct light in the dark section.

  In this modification, in such a case, the calculation result of luminance for each pixel can be made more stable.

  That is, in the image generating apparatus 10 according to the present modification, a partial region (specifically, a selected pixel) of the light receiving surface of the image sensor includes a plurality of light sources including the first light source 141A and the second light source 141B. When each of these sequentially illuminates the substance, an image having the maximum luminance value and an image having the minimum luminance value are acquired for each period. Then, the light / dark image processing unit 120, which is a processing circuit, includes a first image group including a maximum luminance image having a maximum luminance value acquired for each cycle and a luminance having a minimum luminance value acquired for each cycle. Of the second image group including the minimum image, an image group having a small luminance value variance is selected. Here, the bright and dark image processing unit 120 (i) when the selected image group is the first image group, the light and dark image processing unit 120 is an image having the maximum luminance value in the first image group, An image acquired by the image sensor 150 when the one light source 141A is illuminating the substance is selected as the first image. Furthermore, the light / dark image processing unit 120 is an image acquired by the image sensor 150 at a timing shifted by a half cycle of the first image acquired by the image sensor 150, and the second light source 141B An image acquired by the image sensor 150 when the substance is illuminated is selected as the second image. On the other hand, the bright and dark image processing unit 120 (ii) when the selected image group is the second image group, the light and dark image processing unit 120 is an image having the minimum luminance value in the second image group, An image acquired by the image sensor 150 when the light source 141B is illuminating the substance is selected as the second image. Further, the light / dark image processing unit 120 is an image acquired by the image sensor 150 at a timing shifted by a half cycle of the second image acquired by the image sensor 150, and the first light source 141A is An image acquired by the image sensor 150 when the substance is illuminated is selected as the first image. Specifically, each of the first image and the second image has a luminance value corresponding to the same pixel included in the image sensor 150.

  More specifically, the image generation device 10 captures images in a range of three cycles or more, and obtains the maximum value and the minimum value of luminance for each cycle. Hereinafter, the maximum value in one cycle is also referred to as a maximum value in one cycle, and the minimum value in one cycle is also referred to as a minimum value in one cycle. A group having a small variance is selected from among a maximum value group consisting of the maximum value within one period of each period and a minimum value group consisting of the minimum value within one period of each period. If the selected group is a maximum value group, the maximum value in one cycle is selected as the maximum value in all cycles from the plurality of maximum values in one cycle included in the maximum value group, and is selected. This is adopted as the maximum value of the luminance of the pixel. On the other hand, if the selected group is a minimum value group, the minimum value in one cycle is selected as the minimum value in all cycles from the plurality of minimum values in one cycle included in the minimum value group, and is selected. This is adopted as the minimum value of the luminance of the pixel. Here, when the maximum value within the full cycle is selected, the point light source 141 at the illumination position that is separated from the illumination position when the image corresponding to the maximum value within the full cycle is acquired by a distance corresponding to a half cycle is illuminated. Identify the images that were taken when Then, the luminance value of the pixel at the same position as the selected pixel included in the identified image is adopted as the minimum value of the luminance of the selected pixel. On the other hand, when the minimum value within the full period is selected, the point light source 141 at the illumination position that is separated from the illumination position when the image corresponding to the minimum value within the full period is acquired by a distance corresponding to a half period is illuminated. Identify the images taken when you are. Then, the luminance value of the pixel at the same position as the selected pixel included in the identified image is adopted as the maximum luminance value of the selected pixel.

  Hereinafter, a modified example of the first embodiment will be described focusing on differences from the first embodiment. In addition, the configuration of the image generation apparatus 10 according to this modification is the same as that of the first embodiment, and thus detailed description thereof is omitted.

[1. Operation of image generation apparatus]
The operation of the image generation apparatus 10 according to this modification will be described. The overall schematic operation of the image generation apparatus 10 according to this modification is the same as the operation shown in the flowchart shown in FIG. 10 of the first embodiment. However, the detailed operations of the multiple light source imaging in step S1200 and the bright and dark image processing in step S1300 according to this modification are different from those in the first embodiment. Further, the image generation operation in step S1400 according to the present modification is the same as that in the first embodiment, and thus the description of the operation is omitted.

[1-1. Multi-light source shooting]
Details of the multiple light source imaging in step S1200 according to this modification will be described. In step S1200 according to the present modification, among the operations in steps S1210 to S1260 shown in FIG. 11, the operation in step S1210 is different from that in the first embodiment.

  In Embodiment 1, in step S1210, control unit 160 refers to an illumination position list indicating a plurality of predetermined illumination positions, and determines whether or not shooting of an object illuminated from each illumination position has ended. judge. Specifically, the illumination positions of the plurality of point light sources 141 shown in the illumination position list are arranged along the direction crossing the slit, in a two-cycle section, separated from each other by an interval of 1/3 period. In this modification, the illumination positions of the plurality of point light sources 141 shown in the illumination position list are separated from each other by an interval of 1/6 period in a section of 3 periods or more, for example, 5 periods, along the direction crossing the slit. Are arranged. That is, in the illumination position list in the present modification, six illuminations are arranged in one cycle section and in five cycle sections. 31 illumination positions including illuminations arranged at both ends are shown.

  Here, when photographing with illumination from all illumination positions included in the illumination position list has been completed (yes in step S1210), the process proceeds to step S1300. On the other hand, if shooting with illumination from any illumination position in the illumination position list has not been completed (No in step S1210), the process proceeds to step S1220.

  Thus, in each of the three or more periods of illumination, in the present modification, photographing is performed in the bright and dark sections in each of the five illumination periods. Therefore, even when there is unevenness in the light intensity in the light and dark sections, the maximum brightness value in the plurality of bright sections and the minimum brightness value in the plurality of dark sections can be referred to.

[1-2. Bright and dark image processing]
Next, details of the bright and dark image processing in step S1400 according to this modification will be described.

  FIG. 16 is a flowchart showing an example of details of the operation of the bright and dark image processing unit 120 according to the present modification. In the flowchart shown in FIG. 16, steps similar to steps S1301 to S1330, step S1350, and step S1360 shown in FIG. 14 of the first embodiment are denoted by the same reference numerals and detailed description thereof is omitted. To do.

  In step S1310, the data acquisition unit 121 acquires from the storage unit 110 the plurality of images acquired in step S1200 and the illumination positions corresponding to the plurality of images. In step S1320, the pixel selection unit 126 determines whether the luminance value calculation processing has been completed for all pixels included in the image to be generated.

  When the calculation process has been completed for all the pixels included in the generated image (yes in step S1320), the light / dark image processing unit 120 ends the light / dark image process (proceeds to step S1400).

  If the calculation process has not been completed for any pixel included in the generated image (NO in step S1320), the light / dark image processing unit 120 continues the light / dark image process (proceeds to step S1330).

  In step S1330, the pixel selection unit 126 selects, as a selected pixel, one pixel that has not yet been calculated from among a plurality of pixels included in the generated image.

(Step S2301)
The pixel selection unit 126 classifies each of the plurality of images acquired by the data acquisition unit 121 in step S1310 into a plurality of groups having different periods, that is, groups them. That is, the pixel selection unit 126 groups each image of the plurality of illumination positions for one period into one group for each period including the bright section and the dark section based on the illumination positions corresponding to the plurality of images. Classify. Each of the plurality of groups corresponds to a cycle composed of a bright section and a dark section. Note that the start point and end point of the cycle need not be the boundary between the bright and dark sections. In this modification, the predetermined illumination position list referred to in step S1200 indicates a plurality of illumination positions located in a section of three or more periods. In this modification, six illumination positions per cycle located in a 5-cycle section are shown. That is, in step S2301, the plurality of images are grouped into three or more groups, in this modification, five groups. In step S1200, when photographing by illumination from a plurality of illumination positions located in a section corresponding to five cycles is performed, the plurality of images are grouped into five groups in step S2301.

(Step S2302)
Further, the pixel selection unit 126 determines the maximum value of the luminance of the selected pixel and the minimum value of the luminance of the selected pixel for all the groups generated by the grouping in step S2301, that is, for all the periods. It is determined whether or not it has been done. The maximum value of the luminance is the above-described maximum value within one cycle, and the minimum value of the luminance is the above-described minimum value within one cycle. That is, in step S2302, it is determined whether or not the maximum value in one cycle and the minimum value in one cycle of the selected pixel are determined in each of all cycles.

  If the determination of the maximum value and the minimum value of all the groups has been completed (yes in step S2302), the process proceeds to step S2307.

  If the determination of the maximum value and the minimum value of all the groups has not been completed (No in step S2302), the light and dark image processing unit 120 continues the determination process of the maximum value and the minimum value (proceeds to step S2303). This determination process is a process of steps S2303 to S2306.

(Step S2303)
The pixel selection unit 126 determines one period (that is, a group) in which the determination of the maximum value and the minimum value of the luminance of the selected pixel has not yet been performed among all the periods (that is, all groups).

(Step S2304)
The maximum value determination unit 122 compares the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images classified in the determined one period (that is, group). Specifically, the maximum value determination unit 122 selects a plurality of images photographed by a plurality of illuminations arranged in one period section determined in step S2303 among the plurality of images photographed in step S1200. Among them, the luminance value of the pixel located at the same position as the pixel selected in step S1330 is compared. Thereby, the maximum value determination unit 122 determines the maximum value (that is, the maximum value within one cycle) of the luminance of the selected pixel for the one cycle (that is, group).

(Step S2305)
The minimum value determination unit 123 compares the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images classified in the determined one period (ie, group). Specifically, the minimum value determination unit 123 includes a plurality of images captured by a plurality of illuminations arranged in a section of one cycle determined in step S2303 among a plurality of images captured in step S1200. Among them, the luminance value of the pixel located at the same position as the pixel selected in step S1330 is compared. Thereby, the minimum value determination unit 123 determines the minimum value of the luminance of the selected pixel (that is, the minimum value within one cycle) for the one cycle (that is, group).

(Step S2306)
The storage unit 127 stores the maximum luminance value determined in step S2304 and the illumination position of the point light source 141 when the maximum value is obtained by photographing. Further, the storage unit 127 stores the minimum value of the luminance determined in step S2305 and the illumination position of the point light source 141 when the minimum value is obtained by photographing. At this time, the storage unit 127 selects the maximum luminance value, the illumination position corresponding to the maximum luminance value, the minimum luminance value, and the illumination position corresponding to the minimum luminance value in step S2303. It is memorized together with the cycle. Thereafter, the light and dark image processing unit 120 repeatedly executes the processing from step S2302 to step S2306.

  The light / dark image processing unit 120 repeats the processing from step S2302 to step S2306 to obtain the maximum value and the minimum value of the luminance of the selected pixel for each of all the three or more cycles, and the maximum value and the minimum value. Determine the illumination position corresponding to each of the values.

(Step S2307)
The calculation unit 124 obtains the variance of the maximum luminance value for each period stored in step S2306. Further, the calculation unit 124 obtains the variance of the minimum luminance value for each period. Further, the calculation unit 124 compares the variance of the maximum luminance value with the variance of the minimum value.

  If the variance of the minimum luminance value is smaller than the variance of the maximum value in step S2307, that is, if yes in step S2307, the process proceeds to step S2308. If the variance of the minimum brightness value is greater than the variance of the maximum value in step S2307, or if the variance of the minimum value is equal to the variance of the maximum value, that is, if no in step S2307, the process proceeds to step S2310. That is, the calculation unit 124 selects a group having a small variance among a maximum value group composed of the maximum value within one period of each period and a minimum value group composed of the minimum value within one period of each period.

(Step S2308)
The calculation unit 124 determines the smallest value among the minimum values for each period stored in step S2306 as the final minimum luminance value adopted for the selected pixel. This final minimum value of luminance is the above-described minimum value in all cycles. That is, if the group selected in step S2307 is the minimum value group, the calculation unit 124 calculates the minimum value in one cycle from the plurality of minimum values in one cycle included in the minimum value group. While selecting as the minimum value, it is adopted as the minimum value of the luminance of the selected pixel.

(Step S2309)
The calculation unit 124 identifies an illumination position that is shifted by a half-cycle distance from the illumination position corresponding to the minimum value of luminance of the selected pixel determined in step S2308 (that is, the minimum value in all periods). And the calculation part 124 refers to the image image | photographed with the illumination of the point light source 141 of the specified illumination position from the several image which the memory | storage part 127 hold | maintains. The calculation unit 124 determines the luminance value of the pixel located in the same position as the selected pixel included in the referenced image as the final maximum luminance value adopted for the selected pixel. This final maximum value of luminance is the above-described maximum value in all cycles. That is, when the calculation unit 124 selects the minimum value within the entire period in step S2308, the calculation unit 124 sets the point of the illumination position that is separated by a half-cycle distance from the illumination position when the image corresponding to the minimum value within the entire period is acquired. An image taken when the light source 141 is illuminated is specified. Then, the calculation unit 124 employs the luminance value of the pixel at the same position as the selected pixel included in the specified image as the maximum luminance value of the selected pixel.

(Step S2310)
The calculation unit 124 determines the largest value among the maximum values for each period stored in step S2306 as the final maximum luminance value adopted for the selected pixel. This final maximum value of luminance is the above-described maximum value in all cycles. That is, if the group selected in step S2307 is the maximum value group, the calculation unit 124 calculates the maximum value within one cycle from the plurality of maximum values within one cycle included in the maximum value group within all cycles. While selecting as a maximum value, it employ | adopts as a maximum value of the brightness | luminance of a selection pixel.

(Step S2311)
The calculation unit 124 identifies an illumination position that is shifted by a half-cycle distance from the illumination position corresponding to the maximum luminance value of the selected pixel determined in step S2310 (that is, the maximum value in all cycles). And the calculation part 124 refers to the image image | photographed with the illumination of the point light source 141 of the specified illumination position from the several image which the memory | storage part 127 hold | maintains. The calculation unit 124 determines the luminance value of the pixel at the same position as the selected pixel included in the referenced image as the final minimum luminance value adopted for the selected pixel. This final minimum value of luminance is the above-described minimum value in all cycles. In other words, when the calculation unit 124 selects the maximum value within the entire period in step S2310, the calculation unit 124 determines the point of the illumination position that is separated by a half-cycle distance from the illumination position when the image corresponding to the maximum value within the entire period is acquired. An image taken when the light source 141 is illuminated is specified. Then, the calculation unit 124 employs the luminance value of the pixel at the same position as the selected pixel included in the specified image as the minimum luminance value of the selected pixel.

(Step S1350)
The calculation unit 124 subtracts the minimum luminance value of the selected pixel determined in step S2308 or step S2311 from the maximum luminance value of the selected pixel determined in step S2309 or step S2310.

(Step S1360)
The image generation unit 125 stores the difference between the maximum value and the minimum value calculated in step S1350 as the luminance value of the selected pixel.

  The bright and dark image processing unit 120 can generate the luminance values of all the pixels of the image to be generated by repeating steps S1320 to S1360.

  In the modification of the first embodiment, the calculation unit 124 compares the variance of the maximum value obtained for each period with the variance of the minimum value in step S2307, and is stable among the maximum value and the minimum value. The value was determined. However, the calculation unit 124 may obtain the variance of the distance between the illumination positions where the maximum luminance value is obtained in each of the adjacent periods, that is, the variance of the distance between the maximum values. In this case, similarly, the calculation unit 124 obtains the distance between the illumination positions where the minimum luminance value is obtained in each of the adjacent periods, that is, the variance of the distance between the minimum values. Then, the calculation unit 124 may determine the stable value of the maximum value and the minimum value by comparing the variance of the distance between the maximum values and the variance of the distance between the minimum values.

[2. effect]
As described above, the maximum luminance value that should have been obtained by the light that has passed through the light-transmitting portion of the mask 142 and the light that has been directly blocked by the light-shielding portion of the mask 142 should have been obtained. The minimum luminance value may become unstable. This is because of interference or the like when the light passes through the mask 142. However, according to the image generation apparatus 10 according to the modified example of the first embodiment, shooting is performed with the illumination position set so as to include three or more periods including a bright section and a dark section. Then, the stable one of the minimum value and the maximum value of the luminance is selected as a reference, and the luminance of the photographing result at the illumination position shifted by a half cycle from the illumination position corresponding to the reference is used for the calculation. That is, in this modified example, a combination of the maximum value and the minimum value of luminance generated by the period of the mask 142 is used instead of a value that becomes unstable due to interference or the like. Thereby, noise caused by scattered light or refracted light can be reduced, and a clear image can be generated by direct light. That is, even when there is an unstable state such as interference, it is possible to generate luminance by direct light.

(Embodiment 2)
A second embodiment will be described. In the first embodiment, each pixel is photographed in a range of one cycle or more of a bright interval and a dark interval, the maximum value and the minimum value of luminance are determined for each selected pixel, and the minimum value is subtracted from the maximum value. The luminance value of the selected pixel is obtained. Further, in the modification of the first embodiment, shooting is performed in a section of three or more periods, the maximum value and the minimum value of luminance are obtained for each period, and a value with small variance is selected from the maximum value and the minimum value for each period. . If the value with small variance is the maximum value, the maximum value among the maximum values for each cycle is used as a reference, and conversely, if the value with small variance is the minimum value, Treat the smallest value as a reference. A pixel at the same position as the selected pixel in the image photographed when the point light source 141 at the illumination position deviated by a half-cycle distance from the illumination position at the time of obtaining the image having the reference value is illuminated Is used as the minimum value or the maximum value of the brightness of the selected pixel. Thereby, even when the maximum value or minimum value of luminance as shown in FIG. 15 is unstable, it is possible to reduce noise and generate an image of a substance illuminated by direct light.

  In the second embodiment, an image captured without the mask 142 is used, and an image captured without the mask 142 and an image captured with the mask 142 present are used. As a result, even if the maximum value or minimum value of luminance as shown in FIG. 15 is unstable, scattered light or refracted light can be removed. That is, even in the second embodiment, noise can be reduced and an image of a substance illuminated with direct light can be generated. The second embodiment will be described in detail below.

[1. Configuration of Image Generation Device]
FIG. 17 is a functional block diagram of the image generation apparatus 20 according to the second embodiment. As shown in FIG. 17, the image generation apparatus 20 is an apparatus that generates an image of a translucent substance such as a cell, and includes an imaging unit 200, a storage unit 110, a light / dark image processing unit 120, and an output. Unit 130. The photographing unit 200 includes an illuminator 240, a control unit 260, and an image sensor 150. The illuminator 240 includes a first light source 141A, a second light source 141B, a mask 242 and a driving unit 243.

  That is, the image generation device 20 according to the second embodiment includes the storage unit 110, the bright / dark image processing unit 120, and the output unit 130, as with the image generation device 10 according to the first embodiment. However, the image generation device 20 is different from the image generation device 10 of the first embodiment in that the image generation device 20 includes a shooting unit 200 instead of the shooting unit 100. The imaging unit 200 according to the second embodiment includes an image sensor 150 as in the imaging unit 100 according to the first embodiment. However, the imaging unit 200 is different from the imaging unit 100 of the first embodiment in that the imaging unit 200 includes an illuminator 240 instead of the illuminator 140 and includes a control unit 260 instead of the control unit 160. Moreover, the illuminator 240 according to Embodiment 2 includes the first light source 141A and the second light source 141B, similarly to the illuminator 140 of Embodiment 1. However, the illuminator 240 is different from the illuminator 140 of Embodiment 1 in that the illuminator 240 includes a mask 242 instead of the mask 142 and further includes a drive unit 243. Of the constituent elements in the second embodiment, the same constituent elements as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the second embodiment, as in the first embodiment, a plurality of point light sources 141 included in the illuminator 240 with different positions sequentially illuminate a substance that is an object through the mask 242, so that an image sensor is obtained. 150 and the light / dark pattern of light reaching the object are switched. Note that the mask 242 according to the second embodiment has a slit or a checker pattern as in the first embodiment. That is, the mask 242 has a light transmitting portion that transmits light and a light shielding portion that blocks light. Furthermore, in the second embodiment, photographing is performed even in a state where the mask 242 is not present, that is, in a state where all of the light rays of the point light source 141 are not blocked and reach the image sensor 150 and the object. In the second embodiment, a clear image is generated from the imaging result without the mask 242 and the image in the bright section and the image in the dark section.

  The image generation device 20 according to the second embodiment includes the control unit 260, the storage unit 110, the output unit 130, and the drive unit 243, but these components are not essential, and the image generation device 20 It is not necessary to provide the component.

[1-1. Shooting part]
First, the configuration of the photographing unit 200 will be described. The photographing unit 200 includes an illuminator 240, an image sensor 150, and a control unit 260.

  The illuminator 240 according to Embodiment 2 includes a light source group including a plurality of light sources including the first light source 141A and the second light source 141B, a mask 242 and a drive unit 243. The first light source 141A is a point light source that illuminates a substance. The second light source 141B is a point light source that illuminates the substance from a position away from the first light source 141A by a predetermined distance. The mask 242 has a light-transmitting part through which light from the first light source 141A and the second light source 141B is transmitted and a light-shielding part that blocks the light. In addition, the driving unit 243 drives the mask 242.

  Hereinafter, the details of the illuminator 240 will be described.

  FIG. 18 is a diagram schematically illustrating an example of the structure of the illuminator 240 in the second embodiment. Specifically, the illuminator 240 includes a plurality of point light sources 141 including the first light source 141A and the second light source 141B described above, a mask 242 and a driving unit 243.

  Each of the plurality of point light sources 141 of the illuminator 240 sequentially emits light. The plurality of point light sources 141 are arranged at different positions, and irradiates the object with light from different directions through the mask 242.

  The mask 242 has a slit or a checker pattern. That is, the mask 242 has a light transmitting part that transmits light and a light shielding part that blocks light. The drive unit 243 is a mechanism that includes an actuator such as a motor for installing or removing the mask 242. Specifically, the drive unit 243 installs the mask 242 between the image sensor 150 and the first light source 141A and the second light source 141B, or removes the installed mask 242.

  The image sensor 150 has a plurality of pixels, and the above-described substances are arranged thereon. Each pixel of the image sensor 150 is arranged on the light receiving surface, and acquires the intensity (that is, the luminance value) of light emitted from the plurality of point light sources 141. The image sensor 150 acquires a captured image based on the light intensity acquired by each pixel. That is, the image sensor 150 acquires a first image of a substance when illuminated by the first light source 141 </ b> A among the light source group including the plurality of point light sources 141, and the light source group including the plurality of point light sources 141. A second image of the substance is acquired when illuminated by the second light source 141B. Moreover, the image sensor 150 acquires the 4th image of a substance, when illuminated with the light source contained in the light source group. Here, the first image and the second image are acquired via a mask 242 positioned between the image sensor 150 and the first light source 141A and the second light source 141B. The fourth image is acquired without passing through the mask 242.

  The controller 260 controls light irradiation by the plurality of point light sources 141 and photographing by the image sensor 150. Further, the control unit 260 outputs a control signal to the driving unit 243 for installing and removing the mask 242. Specifically, the control unit 260 outputs a signal for installing the mask 242 or a signal for removing the mask 242 to the driving unit 243. In addition, the control unit 260 controls the order in which the plurality of point light sources 141 emit light and the time interval at which the plurality of point light sources 141 emit light.

[1-2. Storage unit]
The storage unit 110 stores the image acquired by the image sensor 150 in association with the illumination position at the time of shooting set by the control unit 260 and information on the presence / absence of the mask 242. The information on the presence / absence of the mask 242 indicates whether the mask 242 is installed between the image sensor 150 and the plurality of point light sources 141 (that is, whether there is a mask 242), or whether the mask 242 is removed between them (that is, mask). 242).

  FIG. 19 is an example of information stored in the storage unit 110. The storage unit 110 stores the ID for identifying the image acquired by the image sensor 150, the illumination position of the point light source 141 that was turned on at the time of shooting, and the information on the presence or absence of the mask 242 in association with each other. “Yes” indicating whether or not the mask 242 is present indicates a state where the mask 242 is installed, and “No” indicates a state where the mask 242 is removed.

[1-3. Bright and dark image processing unit]
The light / dark image processing unit 120 has the same configuration as the light / dark image processing unit 120 of the first embodiment, and is realized by at least one control circuit or processing circuit. The bright and dark image processing unit 120 according to Embodiment 2 selects an image having a large luminance value or an image having a small luminance value from the first image and the second image, and selects the selected image and the fourth image. A fifth image of the substance is generated by deriving a difference based on the image of Here, in the second embodiment, each of the first image, the second image, the fourth image, and the fifth image has a luminance value corresponding to the same pixel included in the image sensor 150.

  More specifically, the light / dark image processing unit 120 is similar to the light / dark image processing unit 120 shown in FIG. 9 of the first embodiment, and includes a data acquisition unit 121, a maximum value determination unit 122, a minimum value determination unit 123, The calculation unit 124, the image generation unit 125, the pixel selection unit 126, and the storage unit 127 are included.

  The data acquisition unit 121 acquires an image used for image processing, that is, a luminance value for each pixel included in the image and an illumination position corresponding to the image from the storage unit 110.

  The pixel selection unit 126 selects, as a selection pixel, a pixel for performing luminance calculation from a plurality of pixels of the image sensor 150, that is, a plurality of pixels in an image to be generated.

  The maximum value determination unit 122 compares the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images stored in the storage unit 110, specifies an image including a pixel having the maximum luminance value, The maximum luminance value is determined as the maximum luminance value of the selected pixel.

  The minimum value determination unit 123 compares the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images stored in the storage unit 110, specifies an image including a pixel having the minimum luminance value, The minimum luminance value is determined as the minimum luminance value of the selected pixel.

  The storage unit 127 stores the image acquired by the data acquisition unit 121 and the illumination position, stores the maximum value determined by the maximum value determination unit 122 and the illumination position in association with each other, and the minimum value determination unit 123 stores it. The determined minimum value and the illumination position are stored in association with each other.

  For each selected pixel of the image to be generated, the calculation unit 124 converts an image captured in the absence of the mask 242 from twice the maximum luminance value of the selected pixel determined by the maximum value determination unit 122. The luminance value of the pixel included at the same position as the selected pixel is reduced. Alternatively, the calculation unit 124 determines, for each selected pixel, the minimum value determining unit 123 from the luminance value of the pixel at the same position as the selected pixel, which is included in the image captured without the mask 242. Subtract twice the minimum value of the selected pixel. Thereby, the luminance value of each pixel of the image to be generated is calculated.

  As described above, the calculation unit 124 selects an image having a large luminance value or an image having a small luminance value from the first image and the second image described above, and selects the selected image and the fourth image. A fifth image of the material is generated by deriving a difference based on it. Each of the first image, the second image, the fourth image, and the fifth image has the luminance value of the same pixel included in the image sensor 150.

  Here, the area ratio of the light transmitting part and the light shielding part in the mask 242 is 1: 1. Therefore, the calculation unit 124 generates a luminance value of the fifth image by deriving a difference between twice the luminance value of the selected image and the luminance value of the fourth image.

  Specifically, when an image having a large luminance value is selected from the first image and the second image, the calculation unit 124 calculates the fourth image from twice the luminance value of the selected image. The luminance value of the fifth image is generated by subtracting the luminance value. Alternatively, when the image having a small luminance value is selected from the first image and the second image, the calculation unit 124 doubles the luminance value of the selected image from the luminance value of the fourth image. Is subtracted to generate the luminance value of the fifth image.

  The image generation unit 125 generates an image including the luminance value of each pixel calculated by the calculation unit 124.

[1-4. Output section]
The output unit 130 is an output device that presents an image generated by the light and dark image processing unit 120, or a unit that outputs the image as electronic data.

[2. Operation of image generation apparatus]
Next, the operation of the image generation apparatus 20 configured as described above will be described. The overall schematic operation of the image generation apparatus 20 according to the second embodiment is the same as the overall schematic operation of the image generation apparatus 10 shown in the flowchart of FIG.

  That is, the image generation method according to Embodiment 2 is an image generation method for generating an image of a light-transmitting substance, and is a light source including a plurality of light sources including the first light source 141A and the second light source 141B. A group, an image sensor 150 and a mask 242 are used. In this image generation method, the following operations (a) to (e) are performed in step S1200, and the following operations (f) and (g) are performed in steps S1300 and S1400. In (a), a mask 242 is disposed between the image sensor 150 and the first light source 141A and the second light source 141B. In (b), when illuminated by the first light source 141A in a state where the mask 242 is disposed, the image sensor 150 acquires a first image of the substance. In (c), when illuminated by the second light source 141B with the mask 242 disposed, the image sensor 150 acquires a second image of the substance. In (d), the mask 242 is removed from between the image sensor 150 and the first light source 141A and the second light source 141B. In (e), the image sensor 150 acquires a fourth image of the substance when illuminated with a light source included in the above-described light source group with the mask 242 removed. In (f), an image having a large luminance value or an image having a small luminance value is selected from the first image and the second image. In (g), a fifth image of the substance is generated by deriving a difference based on the selected image and the fourth image.

[2-1. Multi-light source shooting]
FIG. 20 is a flowchart illustrating an example of details of the operation of the imaging unit 200 of the image generation apparatus 20 according to the second embodiment. Details of the operation of the photographing unit 200 will be described with reference to FIG.

(Step S2210)
The controller 260 outputs a mask removal signal to the driver 243. The driving unit 243 removes, that is, moves the mask 242 of the illuminator 240 from between the plurality of point light sources 141 and the image sensor 150. Thereby, the control unit 260 sets a state in which the light emitted from the plurality of point light sources 141 is not blocked by the mask 242.

(Step S2220)
The control unit 260 outputs a central lighting signal for lighting the point light source 141 directly above the center of the image sensor 150 among the plurality of point light sources 141. The illuminator 240 receives the central lighting signal from the control unit 260, and starts irradiating only the point light source 141 directly above the center of the image sensor 150 among the plurality of point light sources 141.

(Step S2230)
While the object is illuminated by the point light source 141, the image sensor 150 acquires an image formed by light transmitted through the object from the point light source 141. At this time, the light emitted from the point light source 141 does not pass through the mask 242. That is, the light emitted from the point light source 141 reaches the object and the image sensor 150 without being blocked.

(Step S2240)
The control unit 260 outputs the image acquired in step S2230, the illumination position when the image is acquired, and information on the presence / absence of the mask 242 (specifically, “none”) to the storage unit 110. . The storage unit 110 stores the image, the illumination position, and the information on the presence / absence of the mask 242 in association with each other, for example, as illustrated in FIG.

(Step S2250)
The control unit 260 outputs a mask installation signal to the drive unit 243. The driving unit 243 moves the mask 242 of the illuminator 240 and is installed between the plurality of point light sources 141 and the image sensor 150. Accordingly, the control unit 260 sets a state in which light emitted from the plurality of point light sources 141 is irradiated to the object and the image sensor 150 through the mask 242.

  The subsequent operation is the same as the photographing operation of the first embodiment (step S1200 shown in FIG. 11). In the flowchart of FIG. 20, the same operations as those in FIG. 11 of the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

(Step S1210)
The control unit 260 refers to the illumination position list and determines whether or not the photographing of the object illuminated from each illumination position has been completed.

  Here, when photographing with illumination from all illumination positions included in the illumination position list has been completed (yes in step S1210), the process proceeds to step S1300. On the other hand, if shooting with illumination from any illumination position in the illumination position list has not been completed (No in step S1210), the process proceeds to step S1220.

(Step S1220)
The control unit 260 selects an illumination position that has not been illuminated from among a plurality of illumination positions included in the illumination position list, and outputs a control signal to the illuminator 240.

(Step S1230)
The illuminator 240 starts illumination of the object according to the control signal output from the control unit 260 in step S1220. That is, among the plurality of point light sources 141 included in the illuminator 140, the point light source 141 at the illumination position selected in step S1220 starts light irradiation.

(Step S1240)
While the object is illuminated by the point light source 141, the image sensor 150 passes through the mask 242 from the point light source 141 and further acquires an image formed by light transmitted through the object.

(Step S2260)
The control unit 260 associates the image acquired in step S1240, the illumination position at the time of acquiring the image, and information on the presence / absence of the mask 242 (specifically, “Yes”) and outputs them to the storage unit 110. . The storage unit 110 stores an image, an illumination position, and information on the presence / absence of the mask 242 in association with each other.

(Step S1260)
Then, the control part 260 outputs a control signal to the illuminator 240, and stops the illumination to a target object. After step S1260, the process returns to step S1210.

  By repeating the processing from step S1210 to step S1260, the object is sequentially irradiated with light from the point light sources 141 at all the illumination positions included in the illumination position list. The image sensor 150 acquires an image every time the object is irradiated with light through the mask 242.

[2-2. Bright and dark image processing]
Details of the operation of the bright and dark image processing unit 120 in step S1300 will be described.

  FIG. 21 is a flowchart showing an example of the operation of the bright and dark image processing unit 120 according to the second embodiment. In the flowchart shown in FIG. 21, steps S1301 to S1330 shown in FIG. 14 according to the first embodiment and steps similar to those in step S1360 are denoted by the same reference numerals, and detailed description thereof is omitted.

(Step S1310)
The data acquisition unit 121 of the light / dark image processing unit 120 acquires from the storage unit 110 the plurality of images acquired in step S1200, the illumination positions corresponding to each of the plurality of images, and information on the presence or absence of the mask 242. And stored in the storage unit 127.

(Step S1320)
The pixel selection unit 126 determines whether or not the luminance value calculation processing has been completed for all the pixels included in the generated image.

  When the calculation process has been completed for all the pixels included in the generated image (yes in step S1320), the light / dark image processing unit 120 ends the light / dark image process (proceeds to step S1400).

  If the calculation process has not been completed for any pixel included in the generated image (NO in step S1320), the light / dark image processing unit 120 continues the light / dark image process (proceeds to step S1330).

(Step S1330)
The pixel selection unit 126 selects one pixel from a plurality of pixels included in the generated image. One pixel selected here, that is, the selected pixel is a pixel that has not yet been subjected to calculation processing among a plurality of pixels included in the generated image.

(Step S2331)
The maximum value determination unit 122 is located at the same position as the selected pixel included in each of the plurality of images captured in step S1240 when there is the mask 242 for the pixel selected in step S1330 (ie, the selected pixel). Compare the luminance values of the pixels. As a result of the comparison, the maximum value determination unit 122 determines the maximum luminance value as the maximum luminance value of the selected pixel.

(Step S2332)
The calculation unit 124 calculates the pixel of the same position as the selected pixel included in the image captured in step S2230 when there is no mask 242 from twice the maximum luminance value of the selected pixel determined in step S2331. Subtract the luminance value.

(Step S1360)
The image generation unit 125 stores the luminance value calculated in step S2332 as the luminance of the selected pixel.

  The bright and dark image processing unit 120 can generate the luminance values of all the pixels of the image to be generated by repeating steps S1320 to S1360.

  Here, the calculation in step S2332 will be described.

  When the mask 242 is removed, among the light received by any pixel on the image sensor 150, the light that passes through the object from the point light source 141 and goes straight to reach the image sensor 150 is directly expressed as component D. To do. Further, the light refracted or scattered from the point light source to the medium or the object is defined as a global component G. That is, when the mask 242 is removed, the light received by any pixel on the image sensor 150 is D + G.

In the case where the mask 242 is provided and the light in the bright section is ideally irradiated at an arbitrary pixel, the luminance value at the pixel is maximized in the second embodiment. In this case, the direct component from the point light source 141 is equivalent to the direct component D when the mask 242 is removed. However, when the mask 242 is installed, half of the light emitted from the point light source 141 is blocked by the light shielding portion. Note that the area ratio of the light transmitting portion and the light shielding portion in the mask 242 is 1: 1. Therefore, when the mask 242 is installed, the light that is refracted or scattered is reduced by half compared to the case where the mask 242 is removed. That is, when the mask 242 is installed, the global component is equivalent to half of the global component G when the mask 242 is removed. Therefore, when the mask 242 is installed, the light received by any pixel on the image sensor 150 in the bright section is
It becomes.

In the case where the mask 242 is provided and an arbitrary pixel is in an ideal dark section state, the luminance value in the pixel is minimized in the second embodiment. In this case, since the direct component from the point light source 141 is blocked by the mask 242, it is not received. The received light is a global component in which light that has passed through the light-transmitting portion so that it directly reaches the other pixels on the image sensor 150 from the point light source 141 is refracted or scattered and reaches the pixel. That is, since the area ratio between the light transmitting portion and the light shielding portion of the mask 242 is 1: 1, the light received by the pixels in the dark section is a part of the light emitted from the point light source 141. It is a global component that reaches the target pixel by being refracted or scattered. This global component is half of the global component G when the mask 242 is removed. Therefore, when the ideal state of the dark section, that is, when the minimum luminance value is obtained, the light received by the pixel is
It becomes.

  The light due to refraction or scattering becomes noise and blurs the image due to direct light. Therefore, the image can be sharpened by removing the refraction light or scattered light, that is, the global component G. Therefore, in step S2332 shown in FIG. 21 of the second embodiment, in order to directly determine the component D, the luminance value in the bright section is doubled, that is, the maximum luminance value is doubled. Then, as shown in the following (Expression 1), from the double of the maximum value, a component when light is received with the mask 242 removed, that is, a luminance value when photographing without a mask is subtracted.

  Further, the light received in the dark section, that is, the minimum value of luminance is a global component. Therefore, in order to directly determine the component D, as shown in the following (Equation 2), from the luminance value when photographing without a mask, double the light received in the dark section, that is, twice the minimum luminance value. You may subtract.

  FIG. 22 is a flowchart illustrating another example of the operation of the bright and dark image processing unit 120 according to the second embodiment. Specifically, the flowchart of FIG. 22 shows an operation using the subtraction of (Equation 2).

  The flowchart in FIG. 22 includes steps S2341 and S2342 instead of steps S2331 and S2332 in the flowchart in FIG.

(Step S2341)
The minimum value determination unit 123 is in the same position as the selected pixel included in each of the plurality of images captured in step S1240 when there is the mask 242 for the pixel selected in step S1330 (ie, the selected pixel). Compare the luminance values of the pixels. As a result of the comparison, the minimum value determination unit 123 determines the minimum luminance value as the minimum luminance value of the selected pixel.

(Step S2342)
The calculation unit 124 calculates twice the minimum luminance value of the selected pixel determined in step S2341 from the luminance value of the pixel at the same position as the selected pixel included in the image captured in step S2230 when there is no mask. Decrease. Thereby, the luminance value by the direct component of the selected pixel can be generated.

[3. effect]
As described above, according to the image generation apparatus 20 according to the second embodiment, light is projected onto the object and the image sensor 150 through the mask 242 having a repeated pattern of the light shielding portion and the light transmitting portion. Then, for each selected pixel included in the image to be generated, the maximum luminance value of the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images photographed with transmitted light is set. The maximum value of the luminance of the selected pixel is obtained. Further, for each selected pixel included in the image to be generated, the luminance of the pixel at the same position as the selected pixel of the image captured without the mask 242 from twice the maximum value of the luminance of the selected pixel. Decrease the value. Thereby, the image of the target object illuminated by the direct component from which refracted light or scattered light is removed from the light can be generated. Alternatively, for each selected pixel included in the image to be generated, the minimum luminance value among the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images photographed with transmitted light is set. The minimum value of the luminance of the selected pixel is obtained. Further, for each selected pixel included in the image to be generated, the minimum luminance value of the selected pixel is determined from the luminance value of the pixel at the same position as the selected pixel of the image captured without the mask 242. Reduce by 2 times. Thereby, the image of the target object illuminated by the direct component from which refracted light or scattered light is removed from the light can be generated.

  Thus, in Embodiment 2, noise due to scattered light or refracted light can be reduced, and a clear image can be generated by direct light.

(Modification of Embodiment 2)
A modification of the second embodiment will be described. In the second embodiment, the point light source 141 located immediately above the center of the image sensor 150 is used when shooting is performed without the mask 242 used for calculation. However, strictly speaking, the direct component of the light from the point light source 141 varies depending on the relationship between the position of the point light source 141 and the position of the pixel on the image sensor 150. Therefore, as in the second embodiment, when the illumination position corresponding to the image including the pixel having the luminance with the maximum value or the minimum value and the illumination position at the time of shooting without the mask 242 do not necessarily match, The direct component contains an error.

  Therefore, in the modification of the second embodiment, all of the plurality of illumination positions used when photographing is performed in the state where the mask 242 is present is also used for photographing in the state where the mask 242 is absent. In the luminance value calculation process, the selected pixel is obtained by using the luminance of the image with the mask 242 and the luminance of the image without the mask 242 taken when the same illumination position is used. Calculate the luminance value of. Thereby, the error of the direct component due to the mismatch of the illumination position can be eliminated. Hereinafter, a modified example of the second embodiment will be described focusing on differences from the second embodiment.

  Since the configuration of the image generation apparatus 20 according to this modification is the same as that of the second embodiment, description thereof is omitted.

[1. Operation of image generation apparatus]
The operation of the image generation apparatus 20 according to this modification will be described. The overall schematic operation of the image generation apparatus 20 according to this modification is the same as the overall schematic operation of the image generation apparatus 10 shown in the flowchart of FIG. 10 of the first embodiment. However, the detailed operations of the multiple light source imaging in step S1200 and the bright and dark image processing in step S1300 according to this modification are different from those in the first or second embodiment. In addition, the image generation operation in step S1400 according to this modification is the same as that in the first or second embodiment, and thus the description of the operation is omitted.

[1-1. Multi-light source shooting]
FIG. 23 is a flowchart illustrating an example of detailed operation of the imaging unit 200 of the image generation apparatus 20 according to the modification of the second embodiment. That is, FIG. 23 shows an example of detailed operation of multi-light source imaging (step S1200 in FIG. 10) in the modification of the second embodiment. Details of step S1200 will be described with reference to FIG. Among the operations included in the multiple light source photographing of this modification, the same operations as those in FIG. 11 of the first embodiment and FIG. 20 of the second embodiment are denoted by the same reference numerals, and description thereof is omitted.

(Step S2210)
The controller 260 outputs a mask removal signal to the driver 243. The drive unit 243 moves the mask 242 of the illuminator 240 from between the plurality of point light sources 141 and the image sensor 150. Thereby, the control unit 260 sets a state in which the light emitted from the plurality of point light sources 141 is not blocked by the mask 242.

(Step S4210)
The control unit 260 refers to the illumination position list and determines whether or not the photographing of the object illuminated from each illumination position has been completed. This illumination position list is the same as the illumination position list used under the condition where the mask 242 is installed.

  Here, when photographing by illumination from all illumination positions included in the illumination position list has been completed (yes in step S4210), the process proceeds to step S2250. On the other hand, if shooting with illumination from any illumination position in the illumination position list has not been completed (No in step S4210), the process proceeds to step S4220.

(Step S4220)
The control unit 260 selects an illumination position that has not been illuminated from among a plurality of illumination positions included in the illumination position list, and outputs a control signal to the illuminator 240.

(Step S4230)
The illuminator 240, in accordance with the control signal output from the control unit 260 in step S4220, out of the plurality of point light sources 141 included in the illuminator 240, the light from the point light source 141 at the illumination position selected in step S4220. Start irradiation.

(Step S4240)
While the object is illuminated by the point light source 141, the image sensor 150 acquires an image formed by the light emitted from the point light source 141 and transmitted through the object.

(Step S4250)
The control unit 260 associates the image acquired in step S4240, the illumination position when the image is acquired, and information on the presence / absence of the mask 242 (specifically, “none”) and outputs the associated information to the storage unit 110. . The storage unit 110 stores an image, an illumination position, and information on the presence / absence of the mask 242 in association with each other.

(Step S4260)
Then, the control part 260 outputs a control signal to the illuminator 240, and stops the illumination to a target object. After step S4260, the process returns to step S4210.

  After step S2210, the processing from step S4210 to step S4260 is repeated, so that the object is sequentially irradiated with light from the point light sources 141 at all the illumination positions included in the illumination position list. The image sensor 150 acquires an image without the mask 242 every time the object is irradiated with light.

(Step S2250)
After the imaging without the mask 242 is completed, the control unit 260 outputs a mask installation signal to the drive unit 243. The driving unit 243 moves the mask 242 of the illuminator 240 and is installed between the plurality of point light sources 141 and the image sensor 150. Accordingly, the control unit 260 sets a state in which light emitted from the plurality of point light sources 141 is irradiated to the object and the image sensor 150 through the mask 242.

(Step S1210)
The control unit 260 refers to the illumination position list and determines whether or not the photographing of the object illuminated from each illumination position has been completed. This illumination position list is the same as the illumination position list referenced in step S4210.

  Here, when photographing with illumination from all illumination positions included in the illumination position list has been completed (yes in step S1210), the process proceeds to step S1300. On the other hand, if photographing with illumination from any illumination position in the illumination position list has not been completed (No in step S1210), the process proceeds to step S1120.

(Step S1220)
The control unit 260 selects an illumination position that has not been illuminated from among a plurality of illumination positions included in the illumination position list, and outputs a control signal to the illuminator 240.

(Step S1230)
The illuminator 240 starts illumination of the object according to the control signal output from the control unit 260 in step S1220. That is, among the plurality of point light sources 141 included in the illuminator 240, the point light source 141 at the illumination position selected in step S1220 starts light irradiation.

(Step S1240)
While the object is illuminated by the point light source 141, the image sensor 150 passes through the mask 242 from the point light source 141 and further acquires an image formed by light transmitted through the object.

(Step S2260)
The control unit 260 associates the image acquired in step S1240, the illumination position at the time of acquiring the image, and information on the presence / absence of the mask 242 (specifically, “Yes”) and outputs them to the storage unit 110. . The storage unit 110 stores an image, an illumination position, and information on the presence / absence of the mask 242 in association with each other.

(Step S1260)
Then, the control part 260 outputs a control signal to the illuminator 240, and stops the illumination to a target object. After step S1260, the process returns to step S1210.

  By repeating the processing from step S1210 to step S1260, the object is sequentially irradiated with light from the point light sources 141 at all the illumination positions included in the illumination position list. The image sensor 150 acquires an image every time the object is irradiated with light through the mask 242.

[1-2. Bright and dark image processing]
Details of the operation of the bright and dark image processing unit 120 in step S1300 will be described.

  FIG. 24 is a flowchart illustrating an example of the operation of the light / dark image processing unit 120 according to the modification of the second embodiment. In the flowchart shown in FIG. 24, steps similar to those in FIG. 14 of the first embodiment and FIG. 21 of the second embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

(Step S4310)
The data acquisition unit 121 of the light / dark image processing unit 120 acquires from the storage unit 110 the plurality of images acquired in step S1200, the illumination positions corresponding to each of the plurality of images, and information on the presence or absence of the mask 242. And stored in the storage unit 127.

(Step S4320)
The pixel selection unit 126 determines whether or not the luminance calculation process has been completed for all pixels included in the generated image.

  When the calculation process has been completed for all the pixels included in the image to be generated (yes in step S4320), the light / dark image processing unit 120 ends the light / dark image process (proceeds to step S1400).

  If the calculation process has not been completed for any pixel included in the generated image (no in step S4320), the light / dark image processing unit 120 continues the light / dark image process (proceeds to step S4330).

(Step S4330)
The pixel selection unit 126 selects one pixel from a plurality of pixels included in the generated image. One pixel selected here, that is, the selected pixel is a pixel that has not yet been subjected to calculation processing among a plurality of pixels included in the generated image.

(Step S4331)
The maximum value determination unit 122 has the pixel selected in step S4330 (ie, the selected pixel) at the same position as the selected pixel included in each of the plurality of images captured in step S1240 when there is the mask 242. Compare the luminance values of the pixels. As a result of the comparison, the maximum value determination unit 122 determines the maximum luminance value as the maximum luminance value of the selected pixel. Further, the maximum value determination unit 122 specifies the illumination position where the image including the pixel having the maximum luminance value is captured.

(Step S4332)
The calculation unit 124 selects, from the storage unit 127, an image captured without the mask 242 at the illumination position specified in step S4331.

(Step S4333)
The calculation unit 124 subtracts the luminance value of the pixel at the same position as the selected pixel in the image selected in step S4332 from twice the maximum value of the luminance of the selected pixel determined in step S4331. Note that the image selected in step S4332 illuminates the object without the point light source 141 having the same illumination position as the illumination position when the image including the pixel having the maximum luminance value is acquired passing through the mask 242. This is an image taken while

(Step S1360)
The image generation unit 125 stores the luminance value calculated in step S4333 as the luminance value of the selected pixel.

  The bright and dark image processing unit 120 can generate the luminance values of all the pixels of the image to be generated by repeating steps S4320 to S1360.

  In the example shown in FIG. 24, when the point light source 141 at the same illumination position is illuminating as in (Equation 1) above, it is twice the maximum luminance value obtained in the presence of the mask 242. Thus, the brightness of the image taken without the mask 242 was reduced. However, as described above (Equation 2), when the point light source 141 at the same illumination position is illuminating, it was obtained in the state where the mask 242 was present from the brightness of the image taken without the mask 242. You may reduce 2 times the minimum value of a brightness | luminance.

  FIG. 25 is a flowchart illustrating another example of the operation of the light / dark image processing unit 120 according to the modification of the second embodiment. In the flowchart shown in FIG. 25, steps similar to those in FIG. 24 are denoted by the same reference numerals, and detailed description thereof is omitted. In the flowchart shown in FIG. 25, the operations of steps S4341 and S4343 are performed instead of steps S4331 and S4333 of FIG.

(Step S4341)
The minimum value determination unit 123 is the same as the selected pixel included in each of the plurality of images captured in step S1240 when the mask 242 is installed for the pixel selected in step S4330 (ie, the selected pixel). The luminance value of the pixel at the position is compared. As a result of the comparison, the minimum value determination unit 123 determines the minimum luminance value as the minimum luminance value of the selected pixel. Further, the minimum value determination unit 123 specifies the illumination position where the image including the pixel having the minimum luminance value is captured.

(Step S4343)
The calculation unit 124 subtracts twice the minimum luminance value of the selected pixel determined in step S4341 from the luminance value of the pixel at the same position as the selected pixel in the image selected in step S4332. Note that the image selected in step S 4332 illuminates the object without the point light source 141 having the same illumination position as the illumination position when the image including the pixel having the minimum luminance value is acquired passing through the mask 242. This is an image taken while

[2. effect]
As described above, according to the image generation device 20 according to the modification of the second embodiment, light is projected onto the object and the image sensor 150 through the mask 242 having a repeated pattern of the light shielding portion and the light transmitting portion. Then, for each selected pixel included in the image to be generated, the maximum luminance value of the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images photographed with transmitted light is set. The maximum value of the luminance of the selected pixel is obtained. Further, for each selected pixel, a pixel that is illuminated from the same illumination position as when the maximum value of the brightness is obtained by photographing and is in the same position as the selected pixel in an image photographed without the mask 242 Is obtained as a photographing luminance value without a mask. Then, for each selected pixel, the unmasked shooting luminance value corresponding to the selected pixel is reduced from twice the maximum value of the luminance of the selected pixel. Thereby, the image of the target object illuminated by the direct component from which refracted light or scattered light is removed from the light can be generated.

  Alternatively, for each selected pixel included in the image to be generated, the minimum luminance value among the luminance values of the pixels at the same position as the selected pixel in each of the plurality of images photographed with transmitted light is set. The minimum value of the luminance of the selected pixel is obtained. Further, for each selected pixel, a pixel that is illuminated from the same illumination position as when the minimum value of the luminance was obtained by photographing and is in the same position as the selected pixel in an image photographed without the mask 242 Is obtained as a photographing luminance value without a mask. Then, for each selected pixel, twice the minimum luminance value of the selected pixel is subtracted from the unmasked shooting luminance value corresponding to the selected pixel. Thereby, the image of the target object illuminated by the direct component from which refracted light or scattered light is removed from the light can be generated.

  As described above, in the modification of the second embodiment, noise due to scattered light or refracted light can be reduced, and a clear image can be generated by direct light without an error in the illumination position.

<Effect of Embodiment 2>
FIG. 28A and FIG. 28B are images obtained by photographing a state in which a plurality of glass beads having an average diameter of 0.04 mm are hardened with a resin, as in FIGS. 26A and 26B. The image in FIG. 28A is the same as the image in FIG. 26A. The image in FIG. 28B is an image that is captured and synthesized by the method of the second embodiment. The image in FIG. 28B has a larger contrast and clear outline than the image in FIG. 28A.

  FIG. 29 shows the image brightness profile of the image of FIG. 28A and FIG. 28B. In FIG. 29, the luminance profile of the image of FIG. 28A is shown by a solid line, and the luminance profile of the image photographed and synthesized by the method of Embodiment 2 of FIG. 28B is shown by a broken line. The horizontal axis in FIG. 29 is a value indicating the horizontal axis position of the image by the number of pixels, and the vertical axis is a luminance value. The luminance profile shown in FIG. 29 was calculated at the position indicated by the arrow in FIGS. 28A and 28B.

  The high-luminance peak in FIG. 29 shows no significant difference between an image captured without the mask in FIG. 28A and an image captured and synthesized by the method of Embodiment 2 in FIG. 28B. However, for the low-luminance portion in FIG. 29, the luminance of the image (broken line) in FIG. 28B is lower than the luminance (solid line) of the image in FIG. 28A. That is, the difference in luminance of the image (broken line) in FIG. 28B is larger than the difference in luminance (solid line) in the image of FIG. 28A. Therefore, as shown in FIG. 29, it can be confirmed that an image captured and synthesized by the method of Embodiment 2 has a higher contrast than an image captured without a mask.

(Other embodiments)
As described above, the image generation apparatus according to one or a plurality of aspects has been described based on each embodiment and its modifications. However, the present disclosure is not limited to each of these embodiments and its modifications. Absent. As long as it does not deviate from the gist of the present disclosure, various modifications conceivable by those skilled in the art are applied to the above-described embodiments or modifications thereof, and forms constructed by combining components in different embodiments are also disclosed in the present disclosure. It may be included within the range.

  For example, in each of the above-described embodiments and modifications, one pixel included in the image or image sensor 150 to be generated is sequentially selected as a selected pixel, and a luminance value is calculated for each selected pixel. However, instead of sequentially selecting one pixel included in the image or image sensor 150 to be generated, for example, a block composed of 4 × 4 pixels or 8 × 8 pixels may be sequentially selected. In this case, each of the first to fifth images described above is an image corresponding to the same block included in the image sensor 150.

  In the second embodiment and the modification thereof, the mask 242 is moved. However, the mask 242 may be configured not to be moved by configuring the mask 242 with a liquid crystal shutter. That is, the mask 242 is fixed while being installed between the plurality of point light sources 141 and the image sensor 150. The mask 242 has a slit or a checker pattern as described above, but becomes uniformly transparent when a voltage is applied, for example. That is, at this time, the entire mask 242 becomes a translucent portion, and a state optically similar to the state in which the mask 242 is removed from between the plurality of point light sources 141 and the image sensor 150 can be realized. Therefore, in this case, the illuminator 240 does not include the driving unit 243, and the control unit 260 switches between application of a voltage to the mask 242 and stop of the application, thereby installing and removing the mask 242. Similar operations can be performed. Thereby, further size reduction of the image generation apparatus 20 can be achieved.

  In each of the above embodiments and modifications, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software program that implements the image generation apparatus according to each of the above embodiments or each modification performs the steps included in the flowcharts shown in FIGS. 10, 14, 16, and 20 to 25 on a computer. It is a program to be executed.

Further, in this disclosure, all or part of the units and devices, or all or part of the functional blocks in the block diagrams shown in FIGS. 2, 9, and 17 are semiconductor devices, semiconductor integrated circuits (ICs), Alternatively, it may be executed by one or more electronic circuits including LSI (large scale integration). The LSI or IC may be integrated on a single chip, or may be configured by combining a plurality of chips. For example, the functional blocks other than the memory element may be integrated on one chip. Here, it is called LSI or IC, but the name changes depending on the degree of integration and may be called system LSI, VLSI (very large scale integration), or ULSI (ultra large scale integration). A Field Programmable Gate Array (FPGA) programmed after manufacture of an LSI, or a reconfigurable logic device capable of reconfiguring junction relations inside the LSI or setting up circuit partitions inside the LSI can be used for the same purpose.

Further, all or part of the functions or operations of the unit, the apparatus, or a part of the apparatus can be executed by software processing. In this case, the software is recorded on a non-transitory recording medium such as one or more ROMs, optical disks, hard disk drives, etc., and the software is executed when the software is executed by a processor. Causes specific functions in the software to be executed by the processor and peripheral devices. The system or apparatus may comprise one or more non-transitory recording media in which software is recorded, a processor, and required hardware devices such as an interface.

  The present disclosure can be widely used, for example, in an apparatus that generates an image of a cell mass such as a cell in culture or an embryo, and is useful when photographing an object in an incubator.

10, 20 Image generation device 100, 200 Imaging unit 110, 127 Storage unit 120 Bright / dark image processing unit 121 Data acquisition unit 122 Maximum value determination unit 123 Minimum value determination unit 124 Calculation unit 125 Image generation unit 126 Pixel selection unit 130 Output unit 140 240 Illuminator 141 Point light source 142, 242 Mask 160, 260 Control unit 150 Image sensor 243 Drive unit

Claims (20)

An image generation device for generating an image of a light-transmitting substance,
A first light source that illuminates the substance;
A second light source for illuminating the substance from a position away from the first light source by a predetermined distance;
An image sensor on which the substance is disposed;
A light-transmitting portion through which light from the first light source and the second light source is transmitted and a light-shielding portion that blocks the light; and between the image sensor and the first light source and the second light source. A mask located at
A processing circuit,
The image sensor is
Obtaining a first image of the substance when illuminated by the first light source, obtaining a second image of the substance when illuminated by the second light source;
The processing circuit includes:
A difference between a luminance value of a pixel included in the first image and a luminance value of a pixel included in the second image at the same position as the pixel included in the first image. Generating a third image of the substance by deriving,
Image generation device. The partial area of the light receiving surface of the image sensor is
Acquiring the first image in a state where the translucent portion of the mask is disposed between the region and a first light source that illuminates the substance;
Acquiring the second image in a state in which the light-shielding portion of the mask is disposed between the region and a second light source that illuminates the substance;
The image generation apparatus according to claim 1. The partial area of the light receiving surface of the image sensor is
When each of a plurality of light sources including the first light source and the second light source is sequentially illuminating the substance, a plurality of brightness values different from each other including the first image and the second image are included. Get an image,
The processing circuit includes:
Among the plurality of images, an image having the maximum luminance value, which is acquired by the image sensor when the first light source is illuminating the substance, is selected as the first image. And
Among the plurality of images, an image having a minimum luminance value, which is acquired by the image sensor when the second light source illuminates the substance, is selected as the second image. To
The image generation apparatus according to claim 2. The first light source is located half a cycle away from the second light source;
The partial area of the light receiving surface of the image sensor is
When each of a plurality of light sources including the first light source and the second light source sequentially illuminates the substance, an image having a maximum luminance value and an image having a minimum luminance value are acquired.
The processing circuit includes:
Selecting an image group having a small variance in luminance value from among a first image group including a maximum luminance image having a maximum luminance value and a second image group including a minimum luminance image having a minimum luminance value;
(I) When the selected image group is the first image group,
An image having the maximum luminance value in the first image group, the image acquired by the image sensor when the first light source illuminates the substance, the first image Select as
Selecting an image acquired by the image sensor when the second light source is illuminating the substance as the second image;
(Ii) When the selected image group is the second image group,
An image having a minimum luminance value in the second image group, the image acquired by the image sensor when the second light source illuminates the substance, the second image Select as
Selecting an image acquired by the image sensor as the first image when the first light source is illuminating the substance;
The image generation apparatus according to claim 2. An image generation device for generating an image of a light-transmitting substance,
A light source group comprising a plurality of light sources including a first light source that illuminates the substance, and a second light source that illuminates the substance from a position away from the first light source by a predetermined distance;
An image sensor on which the substance is disposed;
A mask having a light-transmitting part through which light from the first light source and the second light source passes, and a light-shielding part that blocks the light;
A processing circuit,
The image sensor is
(A) acquiring a first image of the substance when illuminated by the first light source;
(B) obtaining a second image of the substance when illuminated by the second light source;
(C) acquiring a fourth image of the substance when illuminated by a light source included in the light source group;
Here, the first image and the second image are acquired through the mask positioned between the image sensor, the first light source, and the second light source, and the fourth image is obtained. Is obtained without going through the mask,
The processing circuit includes:
By selecting an image having a large luminance value or an image having a small luminance value from the first image and the second image, and deriving a difference based on the selected image and the fourth image Generating a fifth image of the substance,
Image generation device. Each of the first image, the second image, the fourth image, and the fifth image is
Having a luminance value corresponding to the same pixel included in the image sensor;
The image generation apparatus according to claim 5. The area ratio of the light transmitting portion and the light shielding portion in the mask is 1: 1.
The image generation apparatus according to claim 5 or 6. The processing circuit includes:
Generating a brightness value of the fifth image by deriving a difference between twice the brightness value of the selected image and the brightness value of the fourth image;
The image generation apparatus according to claim 7. The processing circuit includes:
When an image having a large luminance value is selected from the first image and the second image,
Generating the brightness value of the fifth image by subtracting the brightness value of the fourth image from twice the brightness value of the selected image;
The image generation apparatus according to claim 8. The processing circuit includes:
When an image having a small luminance value is selected from the first image and the second image,
Generating a brightness value of the fifth image by subtracting twice the brightness value of the selected image from the brightness value of the fourth image;
The image generation apparatus according to claim 8. An image generation method for generating an image of a light-transmitting substance,
A first light source that illuminates the substance;
A second light source for illuminating the substance from a position away from the first light source by a predetermined distance;
An image sensor on which the substance is disposed;
A light-transmitting portion through which light from the first light source and the second light source is transmitted and a light-shielding portion that blocks the light; and between the image sensor and the first light source and the second light source. And a mask located at
(A) when illuminated by the first light source, the image sensor acquires a first image of the substance;
(B) when illuminated by the second light source, the image sensor acquires a second image of the substance;
(C) a luminance value of a pixel included in the first image, and a luminance value of a pixel included in the second image at the same position as the pixel included in the first image Generating a third image of the substance by deriving a difference of
Image generation method. The partial area of the light receiving surface of the image sensor is
In (a) above,
Acquiring the first image in a state where the translucent portion of the mask is disposed between the region and a first light source that illuminates the substance;
In (b) above,
The image generation method according to claim 11, wherein the second image is acquired in a state where the light shielding portion of the mask is disposed between the region and a second light source that illuminates the substance. In (a) and (b) above,
The partial area of the light receiving surface of the image sensor is
When each of a plurality of light sources including the first light source and the second light source is sequentially illuminating the substance, a plurality of brightness values different from each other including the first image and the second image are included. Get an image,
In (c) above,
Among the plurality of images, an image having the maximum luminance value, which is acquired by the image sensor when the first light source is illuminating the substance, is selected as the first image. And
Among the plurality of images, an image having a minimum luminance value, which is acquired by the image sensor when the second light source illuminates the substance, is selected as the second image. The image generation method according to claim 12. The first light source is located half a cycle away from the second light source;
In (a) and (b) above,
The partial area of the light receiving surface of the image sensor is
Acquiring an image having a maximum luminance value and an image having a minimum luminance value when each of a plurality of light sources including the first light source and the second light source sequentially illuminates the substance. To obtain a plurality of images including the first image and the second image,
In (c) above,
Among the first image group including the maximum luminance image having the maximum luminance value to be acquired and the second image group including the minimum luminance image having the minimum luminance value to be acquired, the luminance value variance is small. Select an image group,
(I) When the selected image group is the first image group,
An image having the maximum luminance value in the first image group, the image acquired by the image sensor when the first light source illuminates the substance, the first image Select as
Selecting an image acquired by the image sensor when the second light source is illuminating the substance as the second image;
(Ii) When the selected image group is the second image group,
An image having a minimum luminance value in the first image group, the image acquired by the image sensor when the second light source illuminates the substance, the second image Select as
The image generation method according to claim 12, wherein an image acquired by the image sensor when the first light source illuminates the substance is selected as the first image. An image generation method for generating an image of a light-transmitting substance,
A light source group comprising a plurality of light sources including a first light source that illuminates the substance, and a second light source that illuminates the substance from a position away from the first light source by a predetermined distance;
An image sensor on which the substance is disposed;
Using a light-transmitting part through which light from the first light source and the second light source passes, and a mask having a light-shielding part that blocks the light,
(A) disposing the mask between the image sensor and the first light source and the second light source;
(B) when illuminated by the first light source with the mask disposed, the image sensor acquires a first image of the substance;
(C) when illuminated by the second light source in a state where the mask is disposed, the image sensor acquires a second image of the substance;
(D) removing the mask from between the image sensor and the first light source and the second light source;
(E) With the mask removed, the image sensor acquires a fourth image of the substance when illuminated by a light source included in the light source group,
(F) selecting an image having a large luminance value or an image having a small luminance value from the first image and the second image;
(G) generating a fifth image of the substance by deriving a difference based on the selected image and the fourth image;
Image generation method. Each of the first image, the second image, the fourth image, and the fifth image is
The image generation method according to claim 15, wherein the image generation method has a luminance value corresponding to the same pixel included in the image sensor. The image generation method according to claim 15 or 16, wherein an area ratio between the light-transmitting portion and the light-shielding portion in the mask is 1: 1. In (g) above,
Generating a brightness value of the fifth image by deriving a difference between twice the brightness value of the selected image and the brightness value of the fourth image;
The image generation method according to claim 17. In (g) above,
When an image having a large luminance value is selected from the first image and the second image,
Generating the brightness value of the fifth image by subtracting the brightness value of the fourth image from twice the brightness value of the selected image;
The image generation method according to claim 18. In (g) above,
When an image having a small luminance value is selected from the first image and the second image,
Generating a brightness value of the fifth image by subtracting twice the brightness value of the selected image from the brightness value of the fourth image;
The image generation method according to claim 18.