JP2016119890A - Image output device, image output method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image output device capable of outputting images suitable for observation of a subject that consists of a plurality of translucent objects which are overlapped with each other three-dimensionally.SOLUTION: An image output device comprises: an illumination part 110 which emits illumination light, which does not substantially contain light flux crossing with each other, to a subject which consists of a plurality of translucent objects which are overlapped with each other three-dimensionally sequentially from a plurality of different illumination positions by using the subject as a reference; an imaging element 120 which acquires a plurality of images of the subject expressed by the illumination light which was emitted from each of the plurality of illumination positions and transmitted through the subject; an image determination part 230 which determines one or more images that the degree of overlapping of the translucent objects in an imaging direction of the images satisfies a predetermined condition based on brightness distribution in the images out of the plurality of images acquired by the imaging element 120; and an output part 300 which outputs the images determined by the image determination part 230.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image output apparatus, an image output method, and a program for imaging a plurality of semitransparent objects that are three-dimensionally overlapping.

  Counting cultured cells is important for screening chemicals and observing cells in clinical settings. Document 1 discloses a technique for automatically counting cells by recognizing the outline of a cell from an image of a sample. Document 2 discloses a method for acquiring information on the three-dimensional structure of a sample from a plurality of images obtained by photographing the sample at different focus positions in the optical axis direction.

JP 2006-079196 A JP, 2014-044050, A

  However, counting of cultured cells is not only performed by a machine, but in clinical situations, a person may count cultured cells based on observation judgment under a microscope. Therefore, there is a need for a technique for presenting a suitable image that can easily obtain an accurate counting result to a machine or a person that performs counting.

  In view of such demands, the disclosed image output apparatus is a microscope image suitable for counting cultured cells, more generally, observation of a subject composed of a plurality of three-dimensionally overlapping semi-transparent objects. Provided is an image output device capable of outputting an image suitable for the above.

  An image output apparatus according to an aspect of the present disclosure is directed to a subject configured by a plurality of semi-transparent objects that are three-dimensionally overlapped with illumination light that substantially does not include mutually intersecting light beams as a reference. An illumination unit that sequentially emits from a plurality of different illumination positions; an imaging device that obtains a plurality of images of the subject represented by the illumination light emitted from each of the plurality of illumination positions and transmitted through the subject; Among the plurality of images acquired by the imaging device, one or more images satisfying a predetermined degree of overlap in the imaging direction of the image of the translucent object based on a luminance distribution in the image. An image determining unit for determining, and an output unit for outputting the image determined by the image determining unit.

  According to the present disclosure, it is possible to obtain an image output device that can output an image suitable for observing a subject formed of a plurality of semi-transparent objects that are three-dimensionally overlapped.

2 is a block diagram illustrating an example of a functional configuration of the image output apparatus according to Embodiment 1. FIG. 3 is a conceptual diagram schematically illustrating an example of a structure of an imaging unit according to Embodiment 1. FIG. 3 is a conceptual diagram schematically illustrating an example of a movable mechanism of an imaging unit according to Embodiment 1. FIG. 6 is a graph illustrating an example of a histogram of luminance values according to the first embodiment. 3 is a flowchart illustrating an example of an operation of the image output apparatus according to the first embodiment. 6 is a conceptual diagram schematically illustrating an example of a structure of an imaging unit according to Modification 1 of Embodiment 1. FIG. It is a figure explaining the fluctuation | variation of the illumination intensity according to an illumination position. It is a figure explaining the fluctuation | variation of the illumination intensity according to an illumination position. FIG. 10 is a block diagram illustrating an example of a functional configuration of an image output apparatus according to a fifth modification of the first embodiment. 10 is a flowchart illustrating an example of an operation of the image output apparatus according to the fifth modification of the first embodiment. 6 is a block diagram illustrating an example of a functional configuration of an image output apparatus according to Embodiment 2. FIG. 10 is a flowchart illustrating an example of the operation of the image output apparatus according to the second embodiment. It is a figure explaining the three-dimensional overlap of a cell. It is a figure explaining the three-dimensional overlap of a cell. It is a figure explaining the three-dimensional overlap of a cell. 10 is a block diagram illustrating an example of a functional configuration of an image output apparatus according to Embodiment 3. FIG. 14 is a flowchart illustrating an example of an operation of the image output apparatus according to the third embodiment. 10 is a diagram illustrating an example of imaging cell information in Embodiment 3. FIG. FIG. 10 is a diagram showing an example of standard cell information in the third embodiment. 14 is a flowchart illustrating an example of an operation of the image output apparatus according to the third embodiment.

(Background to devising one aspect of the present disclosure)
The present inventor has found that the following problems occur with respect to the counting of cultured cells described in the background art.

  The cultured cells may not only spread in one layer but also overlap three-dimensionally like an embryo or a cell mass. Since the depth of field of the microscope is shallow, in the microscopic image obtained by observing a plurality of cells overlapping in the optical axis direction, the outline of those cells may be interrupted and appear to be connected to other cells. is there.

  When performing automatic counting of cells based on the contour line disclosed in Document 1 based on such a microscope image, it is difficult to obtain an accurate counting result.

  This problem occurs not only when automatic counting is performed by a machine, but also when a person counts cells by viewing a microscope image.

  In Document 2, in order to compensate for the shallow depth of field of the microscope, multiple images are acquired at different focus positions, and overlap information (specifically, image feature amounts) is obtained from the change in the image depending on the focus position. A method of configuring is disclosed. However, it is too complicated and practical for a person to compare the number of cells by comparing a plurality of images with different focus positions according to the method.

  Based on the above consideration, the inventors of the present application have devised the following aspects of the invention.

  An image output apparatus according to an aspect of the present disclosure is directed to a subject configured by a plurality of semi-transparent objects that are three-dimensionally overlapped with illumination light that substantially does not include mutually intersecting light beams as a reference. An illumination unit that sequentially emits light from a plurality of different illumination positions; an imaging device that obtains a plurality of images of the subject represented by the illumination light emitted from each of the plurality of illumination positions and transmitted through the subject; Among the plurality of images acquired by the imaging device, one or more images satisfying a predetermined degree of overlap in the imaging direction of the image of the translucent object based on a luminance distribution in the image. An image determining unit for determining, and an output unit for outputting the image determined by the image determining unit.

  Here, the degree of overlap of the translucent object represented in the image varies depending on the image capturing direction, that is, the illumination position. When a person observes the subject in an image in which the degree of overlap of the semi-transparent objects in the imaging direction is large, it may be difficult to determine the shape of each semi-transparent object due to the overlap, making observation difficult. On the other hand, in the automatic counting method for counting the number of the semi-transparent objects based on the luminance distribution in the image, the overlap of the semi-transparent objects represented in the image may be actively used.

  According to this aspect, the overlapping degree of the translucent objects represented in the image among the images of the plurality of subjects represented by the illumination light emitted from each of the plurality of different illumination positions satisfies the predetermined condition 1 The above image is output. The predetermined condition may vary depending on the use of the image. For example, for observation of the subject by a person, an image with a smaller degree of overlapping of the translucent objects is output. In addition, for the automatic counting of the translucent material, an image with a greater degree of overlap of the translucent objects is output. As a result, an image of the subject suitable for the application is output.

  Further, for example, the subject is disposed in a liquid and held in a container together with the liquid, and the image output device is for focusing on the optical path of the illumination light from the illumination unit to the image sensor. Without the optical element, the image output device further includes an imaging start determination unit that determines whether or not a distance between the subject and the imaging element is within a predetermined distance, and the imaging start determination unit After it is determined that the distance between the subject and the image sensor is within a predetermined distance, the illumination unit sequentially emits the illumination light from the plurality of different illumination positions, and the image sensor is configured to emit the illumination light. The plurality of images of the subject represented by

  According to this aspect, the image output apparatus includes a so-called lensless optical system that does not have an optical element for focusing on the optical path of the illumination light from the illumination unit to the imaging element. In the lensless optical system, there is no concept of focus adjustment, and the closer the distance between the subject and the image sensor, the clearer the subject. Therefore, after it is determined that the distance between the subject and the image sensor is within a predetermined distance, the illumination unit sequentially emits the illumination light from the plurality of different illumination positions, and the image sensor The plurality of images of the subject represented by illumination light are acquired.

  Thereby, when the distance between the subject and the image sensor is greater than or equal to the predetermined distance, the illumination unit does not emit illumination light, and the subject is not imaged. That is, it is possible to prevent the blurred image of the subject from being captured due to the distance between the subject and the image sensor.

  In addition, for example, the image output device includes a first time before an imaging start time at which the illumination unit and the imaging element start emission of the illumination light and acquisition of the plurality of images, and the imaging start time. At a second time before and after the first time, the first preliminary image and the second preliminary image of the subject are acquired using the illumination unit and the imaging device, respectively. The imaging start determination unit determines whether the distance between the subject and the imaging element is within the predetermined distance at the second time based on the similarity between the first preliminary image and the second preliminary image. It may be determined whether or not.

  According to this aspect, from the similarity between the first preliminary image and the second preliminary image, for example, by detecting that the subject has reached the bottom of the container and stopped, the subject and the imaging It can be determined that the distance to the element is within the predetermined distance.

  In addition, for example, the image output device uses the illumination unit and the image sensor to reserve the subject before the illumination unit and the image sensor start emitting the illumination light and acquiring the plurality of images. An image is acquired, and the imaging start determination unit determines that the distance between the subject and the imaging element is within the predetermined distance when the kurtosis of the luminance distribution in the preliminary image is equal to or greater than a predetermined threshold. You may judge.

  According to this aspect, by determining that the image of the subject is clear based on the kurtosis of the luminance distribution in the preliminary image, the distance between the subject and the imaging element is within the predetermined distance. It can be determined that there is.

  Further, for example, the imaging start determination unit may determine that the distance between the subject and the imaging element is within the predetermined distance by detecting that the subject has reached the bottom of the container. .

  According to this aspect, when the subject reaches the bottom of the container, it can be determined that the distance between the subject and the imaging element is within the predetermined distance.

  In addition, for example, the subject is held in a container whose posture and position are fixed, the illumination unit is configured to be movable in the plurality of different illumination positions, and the imaging element is configured to move relative to the illumination unit. You may be comprised so that the position which opposes the said illumination part on both sides of the said container is movable, maintaining a attitude | position and a position.

  According to this aspect, even if the posture and position of the illumination unit are moved, the relative positional relationship between the imaging element and the illumination unit does not change. Therefore, it is not necessary to perform a correction process based on the positional relationship between the image sensor and the illumination unit on the captured image.

  Further, for example, the subject is held in a container whose posture and position are fixed, the illumination unit is configured to be movable in the plurality of different illumination positions, and the imaging element sandwiches the container The position and the position may be fixed to face any of the plurality of illumination positions.

  According to this aspect, since the image sensor and the container are fixed, the distance between the image sensor and the container is kept constant even if the illumination unit moves. Therefore, a clear image of the subject can be acquired for any of a plurality of captured images.

  For example, the illumination light may be parallel light or pseudo-coherent light.

  According to this aspect, the illumination light is realized by parallel light or pseudo-coherent light.

  In addition, for example, the image determination unit may determine, from among the plurality of images, a predetermined number of images with a smaller degree of overlapping of the translucent objects in the images as display images.

  According to this aspect, from the plurality of images, up to the predetermined number of images considered suitable for human observation can be determined as the display image.

  In addition, for example, the image determination unit may select up to a predetermined number of images having a smaller dynamic range of luminance in the images, or up to a predetermined number of images having a larger minimum value of luminance in the images, from the plurality of images. May be determined as the display image.

  According to this aspect, the display image can be determined based on a dynamic range of luminance in the image.

  In addition, for example, the image output apparatus may further include a luminance distribution analysis unit that corrects the luminance distribution of the plurality of images according to a change in illuminance on the light receiving surface of the image sensor due to the illumination position.

  According to this aspect, even when the illuminance on the light receiving surface of the image sensor varies according to the illumination position, it is possible to obtain a corrected luminance distribution that can be compared with the plurality of images.

  In addition, for example, the image determination unit determines the display image, and among the plurality of images, the image having the maximum dynamic range of luminance in the image or the minimum value of luminance in the image is minimum. An image is determined as an image for automatic counting, and the image output device further includes a counting unit that counts the number of the translucent objects in the image for automatic counting, and the output unit includes the display image, The number of the translucent objects counted by the counting unit may be output.

  According to this aspect, the number of the translucent objects automatically counted using the image for automatic counting considered to be suitable for the automatic counting process can be output together with the display image.

  Further, for example, the subject is an early embryo, the plurality of translucent objects are a plurality of cells in the early embryo, and the image output device further includes a plurality of growth stages of the early embryo, The standard cell information storage unit storing standard cell information representing the standard number of cells constituting the early embryo and the three-dimensional overlap degree of the cells, and the image determination unit determined by the image determination unit Estimating the growth stage of the initial embryo in the image based on the number of cells in the image, and the degree of overlap represented by the standard cell information for the estimated growth stage and the image of the cells in the image An imaging direction setting unit that sets a limited imaging direction based on the imaging direction of the image when the degree of overlap in the imaging direction matches, and images the initial embryo in a subsequent growth stage Do The illumination unit sequentially emits the illumination light from the illumination positions corresponding to the set imaging direction among the plurality of different illumination positions, and the imaging element is the subject represented by the illumination light. The plurality of images may be acquired.

  According to this aspect, when the subject is an initial embryo, the imaging direction of the initial embryo is limited in the subsequent growth stage with reference to the imaging direction of the image determined from the plurality of images. . As a result, the number of images to be captured is reduced, so that the time required for imaging and analysis of luminance distribution is reduced, and the time required for image output is reduced.

  The imaging direction is limited when the degree of overlap represented by the standard cell information matches the degree of overlap of cells in the image determined from the plurality of images. Thereby, for example, it is inappropriate to limit the imaging direction of the subject in the subsequent growth stage, for example, the overlapping degree of the cells in the image is greatly different from the expected overlapping degree in the estimated growth stage. Under conceivable circumstances, imaging in all directions can be performed without limiting the imaging direction.

  These general or specific aspects may be realized by a recording medium such as a system, method, integrated circuit, computer program, or computer-readable CD-ROM, and the system, method, integrated circuit, computer program. Alternatively, it may be realized by any combination of recording media.

  Hereinafter, an image output apparatus according to an aspect of the present invention will be specifically described with reference to the drawings.

  Note that each of the embodiments described below shows a specific example of the present invention. The numerical values, shapes, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment 1)
In the first embodiment, an image output apparatus that outputs an image obtained by imaging a subject in which a plurality of semitransparent substances such as cell clusters are three-dimensionally overlapped will be described. In the image output apparatus according to Embodiment 1, the degree of overlap of the translucent object in the image satisfies a predetermined condition among a plurality of images obtained by imaging the subject in a plurality of different imaging directions. Output one or more images. The predetermined condition may vary depending on the use of the image. For example, an image with a smaller degree of overlap of the translucent object is output for observing the subject by a person, and the degree of overlap of the translucent object is more for the automatic counting of the translucent material. A large image may be output.

  FIG. 1 is a block diagram illustrating an example of a functional configuration of the image output apparatus according to the first embodiment. The image output device 10 illustrated in FIG. 1 includes an imaging unit 100, a control unit 200, and an output unit 300.

  The imaging unit 100 is a unit that emits illumination light toward the subject and acquires an image of the subject by the illumination light, and includes an illumination unit 110 and an imaging element 120.

  The illumination unit 110 sequentially emits non-interlaced illumination light from a plurality of different illumination positions with respect to the subject toward the subject. Non-intersecting illumination light is illumination light that does not substantially include light beams that intersect with each other, and is light in which the arrival direction of light rays that reach each pixel of the image sensor is uniquely determined. The non-interlaced illumination light can be realized by, for example, parallel light or pseudo-coherent light. The illuminating unit 110 is composed of, for example, a surface light source that restricts the emission direction of the light beam only to the front direction, or a point light source that is combined with a collimating lens or sufficiently separated from the subject, and emits substantially parallel light. It may be emitted. In addition, the illumination unit 110 may be configured by, for example, a monochromatic light source and a pinhole, and emit pseudo coherent light generated by passing the monochromatic light emitted from the monochromatic light source through the pinhole.

  The imaging device 120 acquires a plurality of images of the subject represented by the illumination light emitted from the illumination unit 110 at each of the plurality of illumination positions and transmitted through the subject. The image sensor 120 may be configured with, for example, a CMOS image sensor or a CCD image sensor.

  Here, a structure for the imaging unit 100 to image the subject in a plurality of imaging directions will be described.

  FIG. 2 is a conceptual diagram schematically showing an example of the structure of the imaging unit 100, and shows a schematic structure of the imaging unit 100 viewed from the side and from above.

  In FIG. 2, a cell mass 901 is a subject, and the cell mass 901 is disposed in a culture solution 902 and held in a culture vessel 903 together with the culture solution 902.

  The culture container 903 has a fixed posture and position, the illumination unit 110 is configured to be movable in a plurality of different illumination positions, and the imaging device 120 maintains the posture and position with respect to the illumination unit 110 while maintaining the posture and position. A position facing the illumination unit 110 across the 903 is movably configured.

  In FIG. 2, as an example, the bottom surface of the culture vessel 903 in which the cell mass 901 is held is a horizontal plane. As an example, the illumination position is in the range of 0 degrees to 60 degrees with the vertical direction being 0 degrees in the vertical plane including the center of the bottom surface of the culture container 903, and the specific direction of the culture container 903 is 0 degrees in the horizontal plane. It is provided in increments of 10 degrees over the entire circumference of 0 to 360 degrees. The illumination position is specified by an angle A in the vertical plane and an angle B in the horizontal plane.

  The imaging direction is defined as a direction from the illumination unit 110 toward the image sensor 120. That is, the imaging direction has a one-to-one correspondence with the illumination position. Hereinafter, for convenience, the imaging direction may be represented by an angle A and an angle B that specify the illumination position.

  A movable mechanism of the imaging unit 100 for realizing the above-described movement of the illumination unit 110 and the imaging element 120 will be described.

  FIG. 3 is a conceptual diagram schematically illustrating an example of the movable mechanism of the imaging unit 100, and shows a schematic structure of the imaging unit 100 as viewed from the side.

  The imaging unit 100 illustrated in FIG. 3 is configured by fixing the illumination unit 110 and the imaging element 120 to a frame 130. The frame 130 is held by a support unit 140 that allows rotational movement in at least two directions. The support part 140 may be configured by a ball joint, for example. The drive mechanism 150 rotates the frame 130 so that the illumination unit 110 comes to the designated illumination position when the angle A and the angle B that designate the illumination position are designated.

  The remaining part of the image output apparatus 10 will be described with reference to FIG. 1 again.

  The control unit 200 controls the operation of the imaging unit 100 and processes an image obtained by the imaging unit 100. The imaging control unit 210, the luminance distribution analysis unit 220, the image determination unit 230, and imaging start And a determination unit 240.

  The control unit 200 may be configured by a computer system (not shown) including a CPU, RAM, ROM, and the like, for example. Some or all of the functions of the components of the control unit 200 may be achieved by the CPU executing a program recorded in the ROM using the RAM as a working memory. Further, some or all of the functions of the components of the control unit 200 may be achieved by a dedicated hardware circuit.

  The imaging control unit 210 controls the imaging operation of the subject by the imaging unit 100 by instructing the illumination unit 110 to specify the illumination position and emitting the illumination light, and instructing the imaging device 120 to perform imaging. .

  The luminance distribution analysis unit 220 analyzes the luminance distribution in the image captured by the imaging unit 100. Specifically, the luminance distribution analysis unit 220 may calculate a histogram of luminance values for each pixel in the image and calculate a dynamic range of luminance (difference between the maximum value and the minimum value).

  The image determining unit 230 has a degree of overlap in the imaging direction of the image of the translucent object based on a luminance distribution in the image out of the plurality of images acquired by the imaging element 120 satisfying a predetermined condition. One or more images are determined.

  The degree of overlap of translucent objects is directly defined by the number of translucent objects (also referred to as the number of transmissive layers) that appear to overlap each other in the image capturing direction of the image.

  Further, the degree of overlap of the translucent object is directly defined by the number of transmission layers, and based on the luminance distribution in the image, the width of the dynamic range of the luminance in the image, the luminance in the image, The minimum value may be defined. An image with a wide dynamic range of brightness, that is, an image with a large difference in light and darkness within an image, or an image with a small minimum brightness value, that is, an image with darker dark areas in the image, has more dark areas in the image. This is because it is considered to be formed by overlapping transparent objects.

  The imaging start determination unit 240 determines whether or not the subject and the image sensor 120 are close to each other, that is, whether or not the distance between the subject and the image sensor 120 is within a predetermined distance. The determination will be described in some detail.

  The image output apparatus 10 is configured by a so-called lensless optical system that does not have an optical element for focusing on the optical path of illumination light from the illumination unit 110 to the image sensor 120. In the lensless optical system, there is no concept of focus adjustment, and the closer the distance between the subject and the image sensor 120, the clearer the image can be obtained with less blur due to diffusion and crossing of light due to air or dust in the culture medium. .

  Therefore, in order to find a suitable imaging start time at which a clearer image of the subject can be obtained, the imaging start determination unit 240 determines whether or not the subject and the image sensor 120 are close to each other. This determination is performed, for example, from the preliminary image obtained by preliminarily capturing the subject as follows.

  FIG. 4 is a graph illustrating an example of a histogram of luminance values for each pixel included in a preliminary image of a subject, and a black bar represents a histogram of a clear preliminary image of a subject that is close to the image sensor 120, and is hatched. These bars represent a histogram of a blurred preliminary image of a subject away from the image sensor 120.

  As shown in FIG. 4, whether the subject is close to or away from the image sensor 120 is different in the shape of the histogram depending on whether the preliminary image of the subject is clearly blurred, that is, the luminance distribution is concentrated in the vicinity of the peak. It can be distinguished by whether it spreads gently. Such a difference in the shape of the histogram is reflected, for example, in a statistic called kurtosis around the peak of the luminance distribution. Specifically, in the example of FIG. 4, the kurtosis when the subject is close to the image sensor 120 is about 8, and the kurtosis when the subject is away from the image sensor 120 is about 0.8. .

  The imaging start determination unit 240 detects the subject by the imaging unit 100 via the imaging control unit 210 before the imaging determination unit 230 starts imaging the plurality of images used for determining the one or more images. Preliminary imaging is performed to obtain a preliminary image of the subject. Then, the kurtosis of the luminance distribution in the preliminary image is calculated, and when the calculated kurtosis is equal to or greater than a predetermined threshold, it is determined that the distance between the subject and the image sensor 120 is within the predetermined distance. To do.

  For example, when the kurtosis around the peak of the luminance distribution is 5 or more, the imaging start determining unit 240 determines that the distance between the subject and the image sensor 120 is within a predetermined distance, and the kurtosis is less than 5 In this case, it may be determined that the distance between the subject and the image sensor 120 is greater than the predetermined distance.

  The difference in the shape of the histogram is reflected not only in the kurtosis around the peak of the luminance distribution but also in statistics such as variance and standard deviation. By comparing, it may be determined whether or not the distance between the subject and the image sensor 120 is within a predetermined distance.

  Next, the operation of the image output apparatus 10 configured as described above will be described.

  FIG. 5 is a flowchart illustrating an example of the operation of the image output apparatus 10. As shown in FIG. 5, when the image output process is started, first, the imaging start determination unit 240 determines whether or not the distance between the subject and the image sensor 120 is within a predetermined distance (S1000). ). If it is determined that the distance between the subject and the image sensor 120 is within a predetermined distance (YES in S1000), the imaging control unit 210 determines whether imaging of the subject in all imaging directions has been completed. (S1110).

  If imaging of the subject in all imaging directions has not been completed (NO in S1110), the imaging control unit 210 designates an imaging direction that has not yet been captured to the imaging unit 100 (S1120). The imaging unit 100 moves the illumination unit 110 to the illumination position corresponding to the designated imaging direction, and performs emission of illumination light and imaging of a subject in response to a command from the imaging control unit 210 (S1130).

  The luminance distribution analysis unit 220 acquires the obtained image from the imaging unit 100, and analyzes the luminance distribution in the image (S1140). For example, the luminance distribution analysis unit 220 calculates the difference between the maximum value and the minimum value of the luminance for each pixel in the image as the luminance dynamic range. As described above, the dynamic range of the luminance in the image is an example of the degree of overlap in the image capturing direction of the translucent objects constituting the captured subject.

  The image determination unit 230 compares the dynamic range calculated in the latest step S1140 with the minimum value of the dynamic range calculated so far (S1150). When the latest dynamic range is smaller than the minimum value so far (minimum in S1150), the image determination unit 230 determines the image obtained in the latest step S1130 as a display image (S1160). The display image determined here is a provisional display image until the imaging of the subject in all imaging directions is completed, and the display image has a dynamic range smaller than the minimum value until then. Each time it is determined. The image determination unit 230 stores the determined imaging direction of the display image and the dynamic range of the display image (that is, a new minimum value of the dynamic range).

  When the imaging of the subject in all imaging directions is completed (YES in S1110), the image determination unit 230 designates the imaging direction of the stored display image to the imaging unit 100 via the imaging control unit 210 (S1170). ). The imaging unit 100 moves the illumination unit 110 to an illumination position corresponding to the designated imaging direction, and performs emission of illumination light and imaging of a subject in response to a command from the imaging control unit 210 (S1180). The output unit 300 outputs the image obtained in step S1180 (S1190). The output image is displayed on a display device (not shown), for example, and presented for observing the subject by a person.

  As described above, according to the image output apparatus 10, the degree of overlap of the translucent object represented in the image is the smallest among the plurality of images obtained by imaging the subject in a plurality of different imaging directions. An image is output as a display image. By displaying the output image on the display device, a display image suitable for observing the subject by a person is presented to the observer.

  The image output apparatus 10 can be modified as follows without impairing such effects.

(Modification 1)
In the embodiment, the imaging unit 100 is configured such that the illumination unit 110 and the imaging element 120 move together, but the configuration of the imaging unit is not limited to this example. In the first modification, an example will be described in which the imaging element is fixed in a posture and a position facing any of a plurality of illumination positions, for example, with a container holding a subject interposed therebetween.

  FIG. 6 is a conceptual diagram schematically illustrating an example of the structure of the imaging unit according to Modification 1. A schematic structure of the imaging unit viewed from the side is illustrated. In the imaging unit 101 shown in FIG. 6, unlike the imaging unit 100 in FIG. 3, the orientation and position of the imaging element 121 are fixed. In the imaging unit 101, as in the imaging unit 100, the posture and position of the culture vessel 903 are fixed, and the illumination unit 110 is configured to be movable in a plurality of different illumination positions. In order to realize the movement of the illumination unit 110, the imaging unit 101 can use the same movable mechanism as the imaging unit 100 shown in FIG.

  The imaging unit 101 configured as described above can also capture a subject in a plurality of different imaging directions.

  Note that the illumination unit is not necessarily configured to be movable. For example, the illuminating unit may include a plurality of light sources fixed at corresponding illumination positions by fixing members provided above the subject. Such an illumination unit does not have a movable part, and emits illumination light sequentially from the plurality of light sources.

  It is also possible to configure an imaging unit in which all the postures and positions of the illumination unit, the culture vessel, and the imaging device are fixed. Such an imaging unit rotates the subject by moving the culture solution, and images the subject in a plurality of different imaging directions. The technique of rotating a subject by controlling the movement of the fluid around the subject is described in, for example, Non-Patent Document 1: Yaxiaer Yalikun, Yoshitake Akiyama and Keisuke Morisima, “Multiple Microfluidic Stream 13”. International Conference on Intelligent Robots and Systems (IROS) November 3-7, 2013. It is disclosed in Tokyo, Japan.

(Modification 2)
When the imaging device 121 and the culture vessel 903 are fixed as in the imaging unit 101 illustrated in FIG. 6, the illuminance on the light receiving surface of the imaging device 121 according to the illumination position (that is, the unit on the light receiving surface). The luminous flux per area) varies. In the second modification, a correction process for compensating for such variation will be described.

  FIG. 7A and FIG. 7B are diagrams for explaining fluctuations in illuminance according to the illumination position. As shown in FIG. 7A, when the illumination light is parallel light, the light receiving area increases as the incident angle of the illumination light becomes shallower (L1 <L2), and the illuminance decreases. As shown in FIG. 7B, when the illumination light is pseudo-coherent light, the light receiving area increases as the distance from the illumination unit 110 to the imaging device 121 increases and the incident angle of the illumination light decreases (L3). <L4) Illuminance decreases. Since the luminance of the image is reduced due to the decrease in illuminance, when the illuminance varies according to the imaging direction, it is not possible to directly compare the luminance distributions of a plurality of images with different imaging directions.

  Therefore, the luminance distribution analysis unit 220 corrects the luminance distribution so as to compensate for variations in illuminance according to the imaging direction. For example, the luminance distribution analysis unit 220 holds the correction amount of the luminance distribution for each imaging direction in advance, and analyzes the luminance distribution in the image in consideration of the correction amount according to the imaging direction of the image. Good.

  In addition, the output unit 300 may correct a decrease in luminance or a shape distortion that occurs in the display image according to the imaging direction, and output the corrected display image.

(Modification 3)
In the embodiment, the image determination unit 230 determines an image with the minimum luminance dynamic range as the display image, but the reference for determining the display image is not limited to this example. In Modification 3, an example in which a display image is determined using another criterion will be described.

  As described above, the overlapping degree of the translucent objects represented in the image can be defined by the minimum value of the luminance in the image in addition to the wide dynamic range of the luminance in the image. That is, an image having a larger minimum luminance value in the image may be defined as an image having a smaller degree of overlap of semi-transparent objects.

  Therefore, the image determination unit determines an image having the maximum luminance minimum value as a display image. The display image determined in this way also has a small degree of overlapping of the translucent objects represented in the image and is suitable for observing the subject by a person.

  Note that the display image is not necessarily the only image such as an image with the minimum luminance dynamic range or an image with the maximum luminance minimum value. Up to a predetermined number of images that are considered suitable for observing a subject by a person may be determined as display images.

  When the observer visually confirms the number of cells shown in the image, if the overlapping of the cells in the image is, for example, up to two layers, it is relatively easy to distinguish the overlapping cells. The number is easy to check. However, if cells overlap in three or more layers, it becomes difficult to identify the outline of the cell.

  Therefore, for example, the image determination unit 230 stores in advance the luminance dynamic range (or the minimum luminance value) in an image in which semi-transparent objects overlap at most two layers as a reference value, and compares it with the reference value. Thus, a plurality of images that are considered that two or more semi-transparent objects overlap at most may be determined as display images.

  Thus, the observer can select an image that is most likely to be observed (for example, cell counting) from the determined plurality of display images and use it for observing the subject. For convenience of selection by the observer, the output unit 300 may present thumbnails of the determined plurality of display images to the observer.

(Modification 4)
In the embodiment, the imaging start determination unit 240 determines whether the subject and the image sensor 120 are close to each other based on the luminance distribution of the preliminary image of the subject. The criteria for determining whether or not are close to each other are not limited to this example. In Modification 4, an example will be described in which whether or not the subject and the image sensor 120 are close to each other is determined based on a comparison between two preliminary images obtained by imaging the subject at different times.

  When a subject is placed in a liquid and held in a container together with the liquid, the liquid sinks if the density of the liquid is smaller than the density of the subject, and if the density of the liquid is larger than the density of the subject Float in the liquid. For example, a cell mass as a subject often sinks in a culture solution.

  When the subject sinks into the liquid, the image sensor 120 is disposed below the container, so that the subject sinks into the liquid and when the subject stops at the bottom of the container, the subject is placed on the image sensor 120. It can be determined that they are closest. Whether the subject is moving or stationary in the liquid can be determined based on, for example, the similarity of images obtained by imaging the subject at different times. If the luminance distribution of the image is different, for example, as shown by a black bar in FIG. 4 and a luminance distribution shown by a hatched bar, the subject is considered to be moving.

  Therefore, the imaging start determination unit 240 determines the first time and the first time before starting the imaging of the plurality of images used for the determination of the one or more images by the image determination unit 230. At a subsequent second time, the subject is preliminarily imaged by the imaging unit 100 via the imaging control unit 210 to obtain a first preliminary image and a second preliminary image of the subject. Then, based on the similarity between the first preliminary image and the second preliminary image, it is determined whether or not the distance between the subject and the image sensor is within the predetermined distance at the second time. .

  Whether or not the first preliminary image and the second preliminary image are similar is determined by, for example, the number of pixels for each luminance interval in the luminance histogram of the first preliminary image and the second preliminary image. You may determine by the sum total of a difference being below a predetermined ratio (for example, 5%) with respect to the total number of pixels. Further, the determination may be made by comparing a difference in statistics such as kurtosis value, variance, and standard deviation in the luminance distribution of the first preliminary image and the second preliminary image with a predetermined threshold value. In order to facilitate the determination, the first preliminary image and the second preliminary image may be captured in the same imaging direction and with the same illumination intensity.

  Thus, the blur image of the subject is captured due to the distance between the subject and the image sensor 120 being similar to the first preliminary image and the second preliminary image. Can be prevented on the basis. When the subject floats on the liquid, the same effect can be obtained by arranging the image sensor 120 above the container.

  In addition to using the luminance distribution of the preliminary image, an optical sensor or a weight sensor (not shown) may be used to detect that the subject has reached the bottom of the container.

(Modification 5)
In the first embodiment, the image output apparatus 10 performs an image output process in which the display image is captured again in the stored imaging direction and output after the imaging of the subject in all directions is completed. Such image output processing can be executed using a relatively small memory, but has a time disadvantage because the display image is captured again and output. In addition, there is a concern that an image captured again in the stored imaging direction is not necessarily suitable as a display image due to a change in the state of the subject.

  Therefore, in Modification 5, an image output apparatus that stores all images captured to determine a display image and outputs the determined display image from the stored images will be described.

  FIG. 8 is a block diagram illustrating an example of a functional configuration of an image output apparatus according to the fifth modification. The image output device 11 shown in FIG. 8 is different from the image output device 10 shown in FIG. Is different.

  FIG. 9 is a flowchart showing an example of the operation of the image output apparatus 11. The flowchart shown in FIG. 9 is different from the flowchart of FIG. 5 in that step S1220 for determining the display image and step S1230 for outputting the display image are changed, and step S1210 for storing the captured image is added. The difference is that the step of shooting again is deleted.

  In the following, the operation of the components of FIG. 8 will be described along the flowchart of FIG. In addition, about the component and step already demonstrated, the same code | symbol is attached | subjected and description is abbreviate | omitted suitably and the component and step which are mainly added and changed are demonstrated.

  The image storage unit 400 stores the obtained image every time a subject is imaged in each imaging direction (S1210).

  Each time the image determination unit 231 determines a provisional display image, the image determination unit 231 stores identification information that identifies the most recent image stored in the image storage unit 400 as a display image (S1220).

  After the imaging in all directions is completed (YES in S1110), the image determination unit 231 notifies the output unit 301 of the stored identification information, and the output unit 300 is identified by the notified identification information. The display image is output by referring to the image storage unit 400 (S1230).

  According to the image output device 11 configured as described above, all the images captured to determine the display image are stored, and the determined display image is output from the stored images. As a result, the time disadvantage caused by re-imaging the display image and the concern about the suitability of the display image captured again are eliminated.

(Embodiment 2)
In the second embodiment, an image output apparatus that outputs a result of automatically counting the number of translucent substances constituting a subject in addition to a display image suitable for observing the subject by a person will be described. The image output apparatus according to Embodiment 2 determines the display image, and among the plurality of images, the image having the maximum dynamic range of luminance in the image, or the minimum value of luminance in the image being the minimum The image is determined as an automatic counting image, the number of translucent substances in the determined automatic counting image is counted, and the counting result and the display image are output.

  FIG. 10 is a block diagram illustrating an example of a functional configuration of the image output apparatus according to the second embodiment. The image output device 12 shown in FIG. 10 differs from the image output device 11 shown in FIG. 8 in that the image determination unit 232 and the output unit 302 in the control unit 202 are changed and a counting unit 500 is added.

  FIG. 11 is a flowchart illustrating an example of the operation of the image output apparatus 12. The flowchart shown in FIG. 11 is different from the flowchart of FIG. 9 in that step S1310 for comparing the dynamic range of luminance is changed, step S1320 for determining the image for automatic counting, and the cells shown in the image for automatic counting are counted. The difference is that step S1330 to be executed and step S1340 to output a count value are added.

  Below, operation | movement of the component of FIG. 10 is demonstrated along the flowchart of FIG. In addition, about the component and step already demonstrated, the same code | symbol is attached | subjected and description is abbreviate | omitted suitably and the component and step which are mainly added and changed are demonstrated.

  The image determination unit 232 compares the dynamic range calculated in the latest step S1140 with the minimum value and the maximum value of the dynamic range calculated so far (S1310).

  When the most recent dynamic range is smaller than the minimum value so far (minimum in S1310), the image determination unit 232 stores identification information that identifies the most recent image stored in the image storage unit 400 as a display image (S1220). ). When the most recent dynamic range is larger than the maximum value so far (maximum in S1310), the image determination unit 232 stores identification information that identifies the most recent image stored in the image storage unit 400 as an automatic counting image. (S1320).

  After the imaging in all directions is completed (YES in S1110), the counting unit 500 refers to the automatic counting image represented by the identification information notified from the image determining unit 232 from the image storage unit 400, and the automatic counting is performed. The translucent object represented in the counting image is counted (S1330), and the output unit 302 outputs a count value indicating the number of translucent objects counted by the counting unit 500 (S1340). For example, the count value may be output together with the display image in a manner superimposed on a part of the display image (S1220).

  Here, an example of the cell count in step S1330 will be described using an example of counting cells constituting a standard early embryo in the 8-division period.

  First, the structure of a standard early embryo in the 8-division period will be described. It is known that the early embryo has a structure in which four cells located on a substantially plane are stacked in two stages in the vertical direction. More specifically, the center of the four cells in each stage is located at the apex of a substantially square, and the four cells in the upper stage are in positions where the four cells in the lower stage are translated in the height direction and rotated 45 degrees. In contact with the lower four cells.

  FIG. 12A, FIG. 12B, and FIG. 12C are diagrams illustrating such three-dimensional overlap of cells, and xy in a three-dimensional Cartesian coordinate system of eight cells constituting a standard early embryo in the 8-division period. Projected images on a plane, an xz plane, and a yz plane are shown. That is, FIG. 12A, FIG. 12B, and FIG. 12C schematically represent images obtained when the initial embryo is imaged with parallel light in the z-axis, y-axis, and x-axis directions, respectively.

  In FIG. 12A, FIG. 12B, and FIG. 12C, for convenience, individual cells are represented by spheres, upper cells are indicated by solid lines, and lower cells are indicated by broken lines. Further, the vertical direction is associated with the z-axis, and the centers of the four cells in each stage are arranged at the vertices of squares on the xy plane.

  The luminance value of each pixel in the images of FIGS. 12A, 12B, and 12C is lower as the number of cells that appear to overlap in the imaging direction at the pixel position (the number of the transmissive layers described above) is higher.

  As an example, the path of the light beam incident on the pixels located at the points P and Q in the image of FIG. 12C is indicated by broken lines in FIGS. 12A and 12B. The light beam having the number of transmission layers of 5 is transmitted to the pixel at the point P, which has transmitted a total of five cells, ie, the two cells at the lower stage and the three cells at the upper stage in FIG. 12C. A light beam having two transmissive layers that has passed through the upper two cells in FIG. 12C is incident on the pixel at point Q. Therefore, the luminance of the pixels at the points P and Q becomes the luminance 5 and the luminance 2 which are decreased according to the number of the transmissive layers from the luminance 0 of the background pixel where the light flux of the transmissive layer number 0 that does not transmit the cell is incident. .

  In addition to the background pixel luminance 0 due to the light beam with the transmission layer number 0, four types of luminance due to the light beams with the transmission layer number 1, 2, 3, and 5 are seen in the pixels on the alternate long and short dash line passing through the points P and Q. It is done. Experience has shown that the brightness corresponding to different numbers of transmission layers is discontinuous and can be distinguished relatively clearly.

  FIG. 12C shows a graph schematically showing the correspondence between the pixel position of the pixel located on the one-dot chain line and the luminance, together with the projected image on the yz plane. For the sake of explanation, the graph shows the luminance that appears discontinuously for each section in the y-axis direction, but in an actual image, the luminance that appears discontinuously for each region on the yz plane is observed.

  Now, the counting unit 500 calculates the number of cells shown in such an image as follows, for example.

  First, an area having the next highest luminance after the background is extracted, and from the shape of the extracted area, an area in which one layer of cells is reflected and the number of cells in one layer are estimated. For this estimation, for example, the contour shape of the extracted region is compared with the standard contour shape of a cell (which may be a circle or an ellipse). When the shape of a part or all of the standard outline of the cell matches the outline shape of the extracted region, one of the cells of the one layer is reflected in the standard outline of the cell. It is estimated that Such a comparison is performed on the entire contour of the extracted region, and the number of locations where the contour shape matches is estimated as the number of cells in the one layer.

  Next, the image is corrected so as to remove the cells for the one layer. For this correction, for example, the difference between the background and the luminance of the area is added to the estimated area. As a result, the image is corrected to an image in which the cells for one layer are not shown.

  This process is repeatedly applied to the corrected image until there is no region having a contour that matches the shape of part or all of the standard contour of the cell. The sum of the number of cells estimated at each iteration is calculated as the number of cells in the image.

  In the example of FIG. 12C, the two cells at both ends at the upper stage are removed from the image in the first iteration, and the initial luminance 3 region is corrected to luminance 2. Then, in the next round, three cells are removed from the image, so that no region has a contour that matches the standard contour of the cells.

  According to such a cell counting method, since the cells are counted while removing the cells for each layer on the image, a more accurate counting result is calculated from an image having a larger degree of overlapping of cells. For this reason, the image determination unit 232 determines an image with a greater degree of cell overlap as an image for automatic counting. The image determination unit 232 may determine an image having a larger luminance dynamic range in the image as the automatic counting image, or may determine an image having the minimum luminance minimum value as the automatic counting image. . Furthermore, an image having a larger number of luminance levels in the image may be determined as the automatic counting image. The image for automatic counting determined in this way is useful for obtaining more accurate cell counting results by automatic counting.

  According to the image output device 12 configured as described above, in addition to outputting an image with a minimum degree of overlap of semi-transparent objects represented in the image as a display image, the image output device 12 also represents a half represented in the image. The number of semi-transparent objects that are automatically counted from the image for automatic counting, which is an image having the maximum overlapping degree of transparent objects, is output.

  This helps the observer to count cells by presenting the number of translucent objects automatically counted from an automatic counting image suitable for automatic counting processing together with the display image suitable for observing a subject by a person. can do.

  The specific method of automatic counting is not limited to the above, and other automatic counting methods may be adopted.

(Embodiment 3)
In the third embodiment, when the subject is an early embryo, the image output apparatus suitable for imaging the early embryo for each growth stage of the early embryo and outputting a display image and an automatic counting result of the number of cells. Will be described. The image output device according to the third embodiment limits the imaging range of an image at a subsequent growth stage based on the determination result of the display image and the automatic counting image at a certain growth stage, and thereby the time required for image output To shorten. In order to limit the imaging range of the image, standard cell information indicating the number of standard cells and the three-dimensional overlapping degree of the cells for each growth stage of the early embryo is used.

  FIG. 13 is a block diagram illustrating an example of a functional configuration of the image output apparatus according to the third embodiment. In the image output device 13 shown in FIG. 13, the imaging control unit 213 and the image determination unit 233 in the control unit 203 are changed as compared with the image output device 12 in FIG. 10, and the standard cell information storage unit 250 and the imaging cell information storage are stored. The difference is that a unit 260 and an imaging direction setting unit 270 are added.

  FIG. 14 is a flowchart illustrating an example of the operation of the image output apparatus 13. The flowchart shown in FIG. 14 differs from the flowchart of FIG. 11 in that step S1520 for specifying the imaging direction is changed, step S1400 for setting the imaging direction, and step S1500 for recording the imaging cell information are added. .

  Below, operation | movement of the component of FIG. 13 is demonstrated along the flowchart of FIG. In addition, about the component and step already demonstrated, the same code | symbol is attached | subjected and description is abbreviate | omitted suitably and the component and step which are mainly added and changed are demonstrated.

  The image output process shown in the flowchart of FIG. 14 is executed for each of a plurality of growth stages of the initial embryo as a subject.

  When the image output process is started, the imaging direction setting unit 270 limits the imaging range of the initial embryo in the current image output process using the standard cell information based on the imaging cell information (S1400). Here, the standard cell information is information representing the number of cells constituting the early embryo and the three-dimensional overlap degree of the cells for each growth stage of the standard early embryo. This is information representing the determination result of the display image and the automatic counting image in the image output process. Detailed contents of the process for limiting the imaging range will be described in detail later.

  The imaging control unit 213 determines whether imaging in the imaging direction included in the imaging range set by the imaging direction setting unit 270 has been completed (S1510).

  If imaging of the subject in the imaging direction included in the set imaging range has not been completed (NO in S1510), the imaging control unit 213 includes an imaging direction that is included in the set imaging range and has not yet been imaged. Is designated to the imaging unit 100 (S1520).

  When the imaging in the imaging direction included in the imaging range set by the imaging direction setting unit 270 is completed (YES in S1510), the image determination unit 233 displays imaging cell information that represents the determination result of the display image and the automatic counting image. Is recorded in the imaging cell information storage unit 260.

  The description of the imaging range setting processing by the imaging direction setting unit 270 will be continued.

  First, the contents of imaging cell information and standard cell information will be described.

  FIG. 15 is a diagram illustrating an example of imaged cell information stored in the imaged cell information storage unit 260.

  As shown in FIG. 15, the imaging cell information includes the imaging direction of the display image in the previous image output process, the dynamic range of the luminance of the display image, the imaging direction of the automatic counting image, and the dynamic luminance of the automatic counting image. Includes range and number of cells counted automatically.

  FIG. 16 is a diagram illustrating an example of standard cell information stored in the standard cell information storage unit 250.

  As shown in FIG. 16, the standard cell information includes, for each standard early embryo growth stage, the number of cells constituting the early embryo in the growth stage, the upper and lower layers of the permeation layer, the luminance. Including the upper and lower reference values of the dynamic range.

  Here, the number of permeable layers is a numerical value indicating the number of layers of cells that appear to overlap when observing the initial embryo, and indicates the upper limit layer number and the lower limit layer number according to the observation direction of the initial embryo. The standard cell information in FIG. 16 indicates that, for example, the early embryos in the fourth growth stage (eight division stage) appear to overlap in any number of layers from 2 to 5 layers depending on the observation direction. Yes. In other words, the early embryo in the fourth growth stage (eight division stage) has no observation direction in which no cells overlap, and any cell overlaps more than six layers. There is also no observation direction that can be seen in.

  The upper and lower permeation layer numbers are the knowledge about the three-dimensional arrangement of the cells that make up the early embryo in each growth stage (for example, four cells divided from two cells do not line up in a straight line) This is calculated based on the knowledge that it is spread and positioned in a substantially planar shape.

  The upper limit reference value of the luminance dynamic range is a reference value that represents the dynamic range of the luminance of an image in which cells up to the upper limit number of layers are overlapped, and the lower limit reference value is an overlap of cells up to the lower limit number of layers. This is a reference value representing the dynamic range of the brightness of the captured image.

  The imaging direction is set as follows using these pieces of information.

  FIG. 17 is a flowchart showing detailed processing contents in step S1400 for setting the imaging direction.

  The imaging direction setting unit 270 determines whether or not imaging cell information is stored in the imaging cell information storage unit 260 (S1410). If the imaging cell information is not stored (NO in S1410), the imaging range is set in all directions. This condition is satisfied, for example, when the initial embryo is imaged for the first time in the current image output process.

  If there is imaging cell information (YES in S1410), the imaging direction setting unit 270 refers to the imaging cell information from the imaging cell information storage unit 260.

  The imaging direction setting unit 270 estimates the growth stage of the initial embryo at the time of the previous image output process from the number of cells of the imaging cell information using the standard cell information (S1420).

  In step S1420, the imaging direction setting unit 270, for example, sets the growth stage of the standard cell information corresponding to the number of cells closest to the number of cells indicated by the imaging cell information to the growth stage of the initial embryo at the previous image output processing. It may be estimated.

  The imaging direction setting unit 270 refers to the upper limit reference value and the lower limit reference value of the luminance dynamic range corresponding to the estimated growth stage from the standard cell information (S1430), and determines the maximum luminance dynamic range from the imaging cell information. Reference is made to the value and the minimum value (that is, the dynamic range of the luminance of the automatic counting image and the dynamic range of the luminance of the display image) (S1440).

  The imaging direction setting unit 270 determines whether or not the minimum value matches the lower limit reference value, for example, based on whether or not the minimum value is included in a range of a predetermined ratio (for example, ± 10%) of the lower limit reference value ( S1450). By this determination, it is determined whether or not the display image determined in the previous image output process is an image that overlaps the lower limit number of layers in the growth stage in which cells are estimated (that is, an image that is optimal as a display image). Determined.

  Further, whether or not the maximum value matches the upper limit reference value is determined based on, for example, whether or not the maximum value is included in a range of ± 10% of the upper limit reference value (S1460). By this determination, the image for automatic counting determined in the previous image output process is an image that is shown overlapping with the upper limit number of layers in the growth stage in which cells are estimated (that is, an image that is optimal as an image for automatic counting). It is determined whether or not there has been.

  When the minimum value matches the lower limit reference value and the maximum value matches the upper limit reference value (YES in S1450 and YES in S1460), the imaging direction setting unit 270 determines the display image and the automatic counting image from the imaging cell information. Each imaging direction is referred to (S1470), and the imaging range of the image in the current image output process is limited based on the referenced imaging direction (S1480).

  The imaging direction setting unit 270 limits the imaging range of the current display image to a direction orthogonal to the imaging direction of the previous display image, for example, by taking the cell division direction into account, and for the current automatic counting The imaging range of the image may be limited to a direction orthogonal to the imaging direction of the previous automatic counting image. The imaging range may be limited as a range of a predetermined area including a specific direction.

  When the minimum value does not match the lower limit reference value or the maximum value does not match the upper limit reference value (NO in S1450 or NO in S1460), the imaging direction setting unit 270 sets the imaging range in all directions (S1490). This avoids the disadvantage that the current imaging range is inappropriately limited when the previous display image or the previous automatic counting image is not the optimum image as the display image or the automatic counting image. The

  According to the image output device 13 configured as described above, the time for performing imaging and analysis of luminance distribution is reduced by limiting the imaging range of the initial embryo using information regarding the arrangement of cells of the early embryo. Thus, the time required for image output is shortened.

  The image output apparatus according to one or more aspects of the present invention has been described based on the embodiment, but the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, one or more of the present invention may be applied to various modifications that can be conceived by those skilled in the art, or forms constructed by combining components in different embodiments. It may be included within the scope of the embodiments.

  The present invention can be widely used in apparatuses for imaging and outputting cultured cells and tissues, and images tissues such as cell clusters or embryos in incubators, culture dishes or dishes, culture flasks, culture tubes and the like. This is useful when outputting images obtained in this way. This is particularly useful when confirming the number of cells from an image.

10, 11, 12, 13 Image output device 100, 101 Imaging unit 110 Illumination unit 120, 121 Imaging element 130 Frame 140 Support unit 150 Drive mechanism 200, 201, 202, 203 Control unit 210, 211, 213 Imaging control unit 220, 223 Luminance distribution analysis unit 230, 231, 232, 233 Image determination unit 240 Imaging start determination unit 250 Standard cell information storage unit 260 Imaging cell information storage unit 270 Imaging direction setting unit 300, 301, 302 Output unit 400 Image storage unit 500, 503 Counting unit 901 Cell mass 902 Culture solution 903 Culture vessel

Claims (16)

An illuminating unit configured to sequentially emit illumination light, which is composed of a plurality of three-dimensionally overlapping semi-transparent objects, and which does not substantially include light beams intersecting each other, from a plurality of different illumination positions with respect to the subject. When,
An imaging device that acquires a plurality of images of the subject represented by the illumination light emitted from each of the plurality of illumination positions and transmitted through the subject;
One or more images satisfying a predetermined degree of overlap in the imaging direction of the image of the translucent object based on a luminance distribution in the image from the plurality of images acquired by the imaging device An image determining unit for determining
An output unit that outputs the image determined by the image determination unit;
An image output apparatus comprising: The subject is placed in a liquid and held in a container with the liquid;
The image output device does not have an optical element for focusing on the optical path of the illumination light from the illumination unit to the imaging element,
The image output device further includes an imaging start determination unit that determines whether or not a distance between the subject and the imaging element is within a predetermined distance,
After the imaging start determination unit determines that the distance between the subject and the imaging element is within a predetermined distance, the illumination unit sequentially emits the illumination light from the plurality of different illumination positions, and the imaging An element obtains the plurality of images of the subject represented by the illumination light;
The image output apparatus according to claim 1. The image output device includes a first time before an imaging start time at which the illumination unit and the imaging element start emission of the illumination light and acquisition of the plurality of images, and before the imaging start time. And, at a second time after the first time, the first preliminary image and the second preliminary image of the subject are respectively acquired using the illumination unit and the imaging device,
The imaging start determination unit determines whether the distance between the subject and the imaging element is within the predetermined distance at the second time based on the similarity between the first preliminary image and the second preliminary image. Determine whether or not
The image output apparatus according to claim 2. The image output device acquires a preliminary image of the subject using the illumination unit and the image sensor before the illumination unit and the image sensor start emitting the illumination light and acquiring the plurality of images. And
The imaging start determination unit determines that the distance between the subject and the imaging element is within the predetermined distance when the kurtosis of the luminance distribution in the preliminary image is equal to or greater than a predetermined threshold.
The image output apparatus according to claim 2. The imaging start determination unit determines that the distance between the subject and the imaging element is within the predetermined distance by detecting that the subject has reached the bottom of the container.
The image output apparatus according to claim 2. The subject is held in a container whose posture and position are fixed,
The illumination unit is configured to be movable between the plurality of different illumination positions,
The image sensor is configured to be movable in a position facing the illumination unit with the container interposed therebetween, while maintaining the posture and position with respect to the illumination unit.
The image output apparatus according to claim 1. The subject is held in a container whose posture and position are fixed,
The illumination unit is configured to be movable between the plurality of different illumination positions,
The imaging element is fixed in a posture and a position facing any of the plurality of illumination positions with the container interposed therebetween.
The image output apparatus according to claim 1. The illumination light is parallel light or pseudo-coherent light.
The image output device according to claim 1. The image determining unit determines, as a display image, images from the plurality of images up to a predetermined number in which the degree of overlap of the translucent object in the image is smaller.
The image output device according to claim 1. The image determination unit displays, from the plurality of images, up to a predetermined number of images having a smaller dynamic range of luminance in the images or up to a predetermined number of images having a larger minimum value of luminance in the image. Decide as an image,
The image output device according to claim 9. The image output device further includes a luminance distribution analysis unit that corrects luminance distribution of the plurality of images according to variation in illuminance on the light receiving surface of the image sensor due to the illumination position.
The image output device according to claim 1. The image determination unit determines the display image, and automatically counts an image having the maximum dynamic dynamic range of the image or an image having the minimum minimum luminance value from the plurality of images. Determined as an image for
The image output device further includes:
A counting unit for counting the number of the translucent objects in the automatic counting image;
The output unit outputs the display image and the number of the translucent objects counted by the counting unit.
The image output apparatus according to claim 1. The subject is an early embryo, and the plurality of translucent objects are a plurality of cells in the early embryo,
The image output device further includes:
Standard cell information storage storing standard cell information indicating the standard number of cells constituting the early embryo and the three-dimensional overlap degree of the cells for each of the plurality of growth stages of the early embryo And
Estimating the growth stage of the initial embryo in the image based on the number of cells in the image determined by the image determination unit, and the degree of overlap represented by the standard cell information for the estimated growth stage An imaging direction setting unit that sets a limited imaging direction with reference to the imaging direction of the image when the degree of overlap in the imaging direction of the image of the cell in the image matches,
When imaging the initial embryo in a subsequent growth stage, the illumination unit sequentially emits the illumination light from an illumination position corresponding to the set imaging direction among the plurality of different illumination positions, and the imaging An element obtains the plurality of images of the subject represented by the illumination light;
The image output apparatus according to claim 1. An image output method executed by an image output apparatus including an illumination unit, an image sensor, an image determination unit, and an output unit,
The illumination unit sequentially emits illumination light that does not substantially include light beams intersecting each other from a plurality of different illumination positions toward a subject composed of a plurality of semi-transparent objects that are three-dimensionally overlapped,
The imaging element obtains a plurality of images of the subject represented by the illumination light emitted from each of the plurality of illumination positions and transmitted through the subject;
The degree of overlap in the imaging direction of the image of the translucent object satisfies a predetermined condition based on a luminance distribution in the image from the plurality of images acquired by the imaging element. Determine one or more images,
The output unit outputs the image determined by the image determination unit;
Image output method. The subject is placed in a liquid and held in a container with the liquid;
The image output device does not have a lens for focusing on the optical path of the illumination light from the illumination to the imaging device, and further includes an imaging start determination unit,
The image output method includes:
The imaging start determination unit determines whether or not a distance between the subject and the imaging element is within a predetermined distance;
After determining that the distance between the subject and the imaging element is within a predetermined distance by the imaging start determination unit,
The illumination unit sequentially emits the illumination light from the plurality of different illumination positions,
The imaging device acquires the plurality of images of the subject represented by the illumination light;
The image output method according to claim 14.   A program for causing a computer to execute the image output method according to claim 14.

Images