JP2012039930A - Method, program and apparatus of image processing for culture observation, and method for producing culture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method of image processing for accurately recognizing the area of a medium drop from inside a culture vessel.SOLUTION: The method of image processing includes: applying transmitted illumination in the culture vessel in which the medium drop used for culturing the culture is housed by lighting having a predetermined illumination pattern, and obtaining an observation image wherein the inside the culture vessel transmittedly illuminated is photographed by an imaging device (Step S1); and recognizing the area of the medium drop in the culture vessel according to the observation image wherein the illumination pattern is reflected in the culture vessel (Step S2-S3). A plurality of observation images having different types of the illumination patterns reflected in the culture vessel from each other are obtained when obtaining the observation image, and the area of the medium drop in the culture vessel is recognized according to the plurality of the observation images when recognizing the medium.

Description

  The present invention relates to an image processing means for detecting a region of a medium used for culturing a culture from a culture container, and a method for producing a culture using the image processing means.

  In recent years, with the development of assisted reproduction technology (ART), it has been practiced to observe the growth state of cultured fertilized eggs by in vitro fertilization. A culture microscope is mentioned as an example of the apparatus which observes the condition of cultures, such as a fertilized egg (for example, refer patent document 1). The culture microscope includes a culture apparatus (invecutor) that forms a suitable environment for culturing fertilized eggs, and a microscopic observation system that microscopically observes the state of the fertilized eggs in the culture container housed in the culture apparatus. The observation image of the fertilized egg is acquired at regular intervals, and the user can recognize the fertilized egg by visual observation and automatically perform observation, recording, management, etc. of the fertilized egg. The

  In such an apparatus, when observing the growth state of a fertilized egg in a culture container, it is necessary to first detect the fertilized egg to be observed. As a detection procedure, generally, the entire culture container is photographed. After detecting the medium (medium drop) immersed in mineral oil from the inside of the culture vessel based on the whole observation image, a plurality of microscopic observation images of the medium drop are acquired while changing the observation visual field position in the medium drop. Based on this microscopic observation image, a fertilized egg to be observed that is injected one by one (or a plurality) into the medium drop is discriminated from other foreign substances and detected. Thus, in order to recognize a fertilized egg, microscopic observation in a high magnification field is generally required. However, as the observation magnification increases, the observation field (observation range) becomes narrower. In contrast, a large number of microscopic observation images need to be taken. Therefore, if the medium drop area can be detected in advance by macro observation prior to detection of the fertilized egg by microscopic observation as described above, the area for obtaining the observation image in the microscopic observation is naturally limited, and the entire observation time is reduced. It becomes possible to shorten.

  In the automatic observation of fertilized eggs, as a method for limiting the exploration area of the medium drop, a method for mechanically specifying the exploration area and coloring the phenol added for checking the culture environment are used. There are ways to separate the colors. However, in the method of mechanically specifying the exploration area, since the user manually creates a medium drop in the dish, the exploration can only be performed by specifying the initial exploration area almost on the entire dish surface, and the effect of limiting the area is effective. I can't get it. In addition, although the color separation method can efficiently limit the search area, the color strength after coloring is unknown depending on the environment, and depending on the user, observation or the like may be performed without phenol. In addition, there are not a few users who write on the upper or lower surface of the dish with a pen. In the extraction by color separation, the detection accuracy of the medium drop region changes depending on the color density or the presence or absence of writing.

JP 2004-229619 A

  Therefore, a measure for extracting a medium drop region from an observation image in a wide observation field of view without depending on color information is desired.

  The present invention has been made in view of such a problem, and is a culture observation image processing method, an image processing program, an image processing apparatus, and a culture that can accurately identify a medium region from a culture vessel. It aims at providing the manufacturing method of.

  In order to achieve such an object, according to an embodiment illustrating the present invention, the inside of a culture container containing a medium used for culture cultivation is illuminated with illumination light having a predetermined illumination pattern, and the transmission is performed. Obtaining an observation image obtained by photographing the inside of the culture vessel that has been illuminated with an imaging device, identifying the medium in the culture vessel based on the observation image in which the illumination pattern is reflected in the culture vessel, and When obtaining an observation image, obtain a plurality of observation images having different forms of the illumination pattern reflected in the culture vessel, and identify the culture medium based on the plurality of observation images in the culture vessel. A culture observation image processing method is provided, wherein the culture medium is identified.

  According to another aspect of the present invention, there is provided an image processing program that causes a computer to function as an image processing apparatus that can be read by a computer and that acquires an image captured by an imaging apparatus and performs image processing. And illuminating the inside of the culture vessel containing the medium used for culturing the culture with the illumination light having a predetermined illumination pattern, and obtaining an observation image obtained by photographing the inside of the culture vessel that has been illuminated by the imaging device A step of identifying the culture medium in the culture container based on the observation image in which the illumination pattern is reflected in the culture container, and a step of outputting the identification result of the culture medium. And in the step of acquiring the observation image, the illumination pattern reflected in the culture vessel In the step of acquiring a plurality of observation images having different states and identifying the culture medium, the culture observation image processing is characterized in that the culture medium in the culture vessel is identified based on the plurality of observation images. A program is provided.

  In addition, according to an embodiment of the present invention, the culture vessel that has been transmitted and illuminated is transmitted through the inside of a culture vessel that contains a medium used for culture culture with illumination light having a predetermined illumination pattern. Based on the observation image obtained by reflecting the illumination pattern in the culture vessel acquired by the input unit, an input unit that acquires an observation image obtained by imaging the inside with the imaging device, the medium in the culture vessel An image analysis unit for identifying, and an output unit for outputting the identification result of the culture medium by the image analysis unit, wherein the input unit includes a plurality of observation images having different forms of the illumination pattern reflected in the culture vessel A culture observation image processing apparatus characterized in that the image analysis unit identifies the medium in the culture container based on the plurality of observation images. That.

  Further, according to an embodiment illustrating the present invention, the culture is cultured under predetermined environmental conditions, and the image processing apparatus according to the present invention is used from the culture container in which the culture medium in which the culture is cultured is stored. A method for producing a culture is provided that identifies the medium.

  Further, according to an embodiment illustrating the present invention, a culture is cultured under predetermined environmental conditions, and the inside of the culture container in which the culture medium in which the culture is cultured is accommodated is transmitted with illumination light having a predetermined illumination pattern. Then, an observation image obtained by imaging the inside of the culture vessel that has been illuminated by the imaging device is acquired, and the culture medium in the culture vessel is obtained based on the observation image in which the illumination pattern is reflected in the culture vessel. When acquiring the observation image, acquiring a plurality of observation images having different illumination patterns reflected in the culture vessel, and identifying the culture medium based on the plurality of observation images A method for producing a culture is provided that identifies the medium in the culture vessel.

  According to the present invention, it is possible to accurately identify a medium region from a culture container.

It is a flowchart which shows the outline | summary of an image processing program. It is a general | schematic block diagram of the culture observation system shown as an example of application of this invention. It is a block diagram of the said culture observation system. (A) is a top view of a culture container, (B) is a perspective view which shows a dish. (A) is a schematic block diagram of a 1st illumination part, (B) is a schematic diagram which shows the stripe pattern displayed on the 1st illumination part. It is a schematic diagram which shows the process which produces | generates a synthesized image. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus. It is a flowchart which shows the detail of step S2 in an image processing program. It is a flowchart which shows the outline | summary of the manufacturing method of a fertilized egg. It is a schematic block diagram which shows the modification of a 1st illumination part, (A) shows the state in which a culture container is located in a lower position, (B) shows the state in which a culture container is located in an upper position.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. As an example of a system to which the image processing apparatus according to the present embodiment is applied, a schematic configuration diagram and a block diagram of a culture observation system are shown in FIGS. 2 and 3, respectively.

  The culture observation system BS roughly observes the culture chamber 2 provided in the upper part of the housing 1, the shelf-like stocker 3 that accommodates and holds a plurality of culture containers 10, and the sample in the culture container 10. An observation unit 5, a transfer unit 4 for transferring the culture vessel 10 between the stocker 3 and the observation unit 5, a control unit 6 for comprehensively controlling the operation of the system, and an operation panel 7 provided with an image display device. Etc.

The culture chamber 2 is a chamber for forming a culture environment, and is kept sealed after the sample is charged in order to prevent environmental changes and contamination. Along with the culture chamber 2, a temperature adjustment device 21 that raises and lowers the temperature in the culture chamber 2, a humidifier 22 that adjusts humidity, and a gas supply device 23 that supplies a gas such as CO ₂ gas or N ₂ gas. A circulation fan 24 for making the entire environment of the culture chamber 2 uniform, an environmental sensor 25 for detecting the temperature, humidity, carbon dioxide concentration, etc. of the culture chamber 2 are provided. The operation of each device is controlled by the control unit 6, and the culture environment defined by the temperature, humidity, carbon dioxide concentration, etc. of the culture chamber 2 is maintained in a state that matches the culture conditions set on the operation panel 7.

  The stocker 3 is formed in a shelf shape that is partitioned into a plurality of parts in the front-rear direction and the vertical direction in FIG. A unique address is set for each shelf. For example, when the front-rear direction is A to C rows and the vertical direction is 1 to 7 rows, the A-row 5 rows shelf is set as A-5. The

  As the culture vessel 10, an appropriate one such as a flask, dish, or well plate is selected according to the type and purpose of the culture. In this embodiment, as shown in FIG. A configuration including five formed dishes 11 (consisting of a container 11a and a lid 11b) having a diameter of about 35 mm and a holder 12 that holds the dish 11 is illustrated. As shown in FIG. A fertilized egg J as a culture is injected into each dish 11 together with a medium drop D containing a pH indicator such as phenol red that changes color according to pH. On the bottom surface of the dish 11, one or more medium drops D of about 20 [μl] dropped by a pipette or the like are formed (three are shown in FIG. 4B). It is in a state immersed in the colorless and transparent mineral oil O inside. In each medium drop D, for example, one (or more) fertilized eggs J collected at the same time from the same mother for external fertilization are inserted. The culture vessel 10 is assigned a code number and is stored in association with the designated address of the stocker 3.

  The transfer unit 4 is provided inside the culture chamber 2 so as to be movable in the vertical direction and is moved up and down by the Z-axis drive mechanism. The transfer unit 4 is attached to the Z stage 41 so as to be movable in the front-rear direction and is moved by the Y-axis drive mechanism. The Y stage 42 that is moved back and forth, the X stage 43 that is attached to the Y stage 42 so as to be movable in the left-right direction and is moved left and right by the X-axis drive mechanism, and the like, is supported by lifting the culture vessel 10 to the tip side of the X stage 43 A support arm 45 is provided. The transport unit 4 has a moving range in which the support arm 45 can move between the entire shelf of the stocker 3 and the observation unit 5. The X-axis drive mechanism, the Y-axis drive mechanism, and the Z-axis drive mechanism are configured by, for example, a servo motor with a ball screw and an encoder, and the operation thereof is controlled by the control unit 6.

  The observation unit 5 includes a first illumination unit 51 that illuminates the sample from the lower side of the sample stage 15, a second illumination unit 52 that illuminates the sample from above the sample stage 15 along the optical axis of the microscopic observation system, and a sample from below. 3, a macro observation system 54 that performs macro observation of the sample, a micro observation system 55 that performs micro observation of the sample, an image processing apparatus 100 (see FIG. 2), and the like. The sample stage 15 is made of a light-transmitting material and has a transparent window 16 in the observation area. The sample stage 15 is composed of a fine drive stage that can be moved in the XY direction (horizontal plane direction) and the Z direction (vertical direction) by operation control from the control unit 6, and the culture vessel 10 placed on the upper surface thereof. Is moved in the XY directions, so that the culture vessel 10 can be inserted on the optical axis of the macro observation system 54 or on the optical axis of the microscopic observation system 55.

  The first illumination unit 51 is composed of a transmissive liquid crystal panel with a backlight provided on the lower frame 1 b side, and illuminates the entire culture vessel 10 from the lower side of the sample stage 15. As shown in FIGS. 5A and 5B, the first illumination unit 51, which is a transmissive liquid crystal panel, has a binary pattern (in this embodiment, a periodic pattern in which bright portions and dark portions are repeatedly arranged as illumination patterns). Display a stripe pattern having a typical stripe pattern). In this state, a striped surface light source is formed on the first illumination unit 51.

  The stripe pattern of the surface light source is sufficiently fine as compared with the diameter of the medium drop D (for example, 6 to 7 mm), and at least two periods of stripes are arranged in the diameter size of the medium drop D. The period of the stripe pattern (the stripe pitch of the surface light source) is set to about 1 mm, for example. The stripe pitch of the surface light source can be controlled by the control unit 6 and can be varied within a predetermined range (for example, within a range of 0.5 to 2 mm). Also, the phase of the stripe pattern is variable under the control of the control unit 6.

  The second illumination unit 52 includes a light source such as an LED and an illumination optical system including a phase ring, a condenser lens, and the like. The second illumination unit 52 is provided in the culture chamber 2 and receives light from the microscopic observation system 55 from above the sample stage 15. The sample in the culture vessel 10 is illuminated along the axis. The third illumination unit 53 superimposes light emitted from each of the light sources, such as a plurality of LEDs and mercury that emit light having a wavelength suitable for epi-illumination observation and fluorescence observation, on the optical axis of the microscopic observation system 55. An illumination optical system composed of a beam splitter, a fluorescent filter, and the like to be disposed in the lower frame 1b located below the culture chamber 2, and the light of the microscopic observation system 55 from below the sample stage 15 The sample in the culture vessel 10 is illuminated along the axis.

  The macro observation system 54 includes an observation optical system 54 a and an imaging device 54 c such as a CCD camera that captures an image of the sample imaged by the observation optical system 54 a and is positioned above the first illumination unit 51. Provided in the culture chamber 2. The macro observation system 54 captures an image of transmitted light of the sample (culture vessel 10) backlit by the first illuminating unit 51, and acquires the entire observation image (macro image) as a color image.

  The microscopic observation system 55 includes an observation optical system 55a composed of an objective lens, an intermediate zoom lens, a fluorescent filter, and the like, and an imaging device 55c such as a cooled CCD camera that takes an image of a sample imaged by the observation optical system 55a. And disposed inside the lower frame 1b. The second illumination unit 52 and the microscopic observation system 55 constitute a phase difference observation microscope. A plurality of objective lenses and intermediate zoom lenses are provided, and are configured to be set to a plurality of magnifications using a displacement mechanism such as a revolver or a slider (not shown in detail). In this embodiment, at least two magnifications for low-magnification observation and high-magnification observation are switched so that magnification can be changed. The microscopic observation system 55 displays the sample in the culture vessel 10 such as a phase difference image by the transmitted light of the sample illuminated by the second illumination unit 52 and a fluorescence image by the fluorescence emitted from the sample illuminated by the third illumination unit 53. A microscopic observation image (micro image) is taken.

  The image processing apparatus 100 performs A / D conversion on signals input from the imaging device 54c of the macro observation system 54 and the imaging device 55c of the microscopic observation system 55, and performs various image processing to perform an entire observation image or a microscopic observation image. Image data is generated. Further, the image processing apparatus 100 performs image composition and image analysis on the image data of these observation images (entire observation image and microscopic observation image), and identifies the medium drop D and the fertilized egg J. Specifically, the image processing apparatus 100 is constructed by executing an image processing program stored in the ROM of the control unit 6 described below. The image processing apparatus 100 will be described in detail later.

  The control unit 6 includes a CPU 61 that executes processing, a ROM 62 that stores and stores control programs and control data of the culture observation system BS, a RAM 63 that temporarily stores observation conditions and image data, and the like. Control operation. Therefore, as shown in FIG. 3, the constituent devices of the culture chamber 2, the transport device 4, the observation unit 5, and the operation panel 7 are connected to the control unit 6. In the RAM 63, the environmental conditions of the culture chamber 2 according to the observation program, the observation schedule, the observation type and observation position in the observation unit 5, the observation magnification, and the like are set and stored. Further, the RAM 63 is provided with an image data storage area for recording image data photographed by the observation unit 5, and index data including the code number of the culture vessel 10 and photographing date / time are associated with the image data and recorded. Is done.

  The operation panel 7 is provided with an operation panel 71 provided with input / output devices such as a keyboard and a switch, and a display panel 72 for displaying an operation screen, image data, and the like. An operation command or the like is input. The communication unit 65 is configured in accordance with a wired or wireless communication standard, and data can be transmitted to and received from a computer or the like externally connected to the communication unit 65.

  In the culture observation system BS schematically configured as described above, the CPU 61 controls the operation of each part based on the control program stored in the ROM 62 according to the setting conditions of the observation program set on the operation panel 7, and the culture vessel 10 The sample inside is automatically captured. That is, when the observation program is started by a panel operation on the operation panel 71 (or a remote operation via the communication unit 65), the CPU 61 reads each condition value of the environmental conditions stored in the RAM 63, and from the environment sensor 25. The environmental state of the culture chamber 2 to be input is detected, and the temperature adjustment device 21, the humidifier 22, the gas supply device 23, the circulation fan 24, etc. are operated according to the difference between the condition value and the actual measurement value. Feedback control is performed on the culture environment such as temperature, humidity, and carbon dioxide concentration.

  Further, the CPU 61 reads the observation conditions stored in the RAM 63 and operates the X, Y, Z stages 41, 42, 43 of the transport unit 4 based on the observation schedule to observe the culture vessel 10 to be observed from the stocker 3. The sample is transported to the sample stage 15 of the unit 5 and observation by the observation unit 5 is started. For example, when the observation set in the observation program is macro observation, the culture vessel 10 transported from the stocker 3 by the transport unit 4 is positioned on the optical axis of the macro observation system 54 and placed on the sample stage 15. Then, the light source of the first illumination unit 51 is turned on to irradiate illumination light (transmitted light) from below the culture vessel 10, and an entire observation image is taken by the imaging device 54c from above the backlight-illuminated culture vessel 10. To do. The signal input from the imaging device 54c to the control unit 6 is processed by the image processing device 100 to generate a whole observation image, and the image data is stored in the image data storage area of the RAM 63 together with index data such as the shooting date and time. Is done.

  At this time, as shown in FIG. 6A, the entire observation image photographed and acquired by the imaging device 54c reflects the stripe pattern of the surface light source. As shown in FIGS. 4B and 5A, since the interface between the culture medium drop D and the mineral oil O is a curved surface, the culture medium drop D exerts a lens action on the passing light flux. For this reason, as shown in FIG. 6A, the stripe pitch of the stripe pattern appearing in the region corresponding to the medium drop D in the entire observation image is narrower than the stripe pitch of the stripe pattern appearing in the other region. As a result, an image of the medium drop D is visualized in the entire observation image.

  At this time, the first illuminating unit 51, which is a transmissive liquid crystal panel, changes the light emission position by controlling the operation of the control unit 6 and changes the phase of the stripe pattern (for example, every ¼ period), thereby culturing. The container 10 is illuminated with a plurality of preset stripe patterns (for example, four types of stripe patterns whose phases are different by ¼ period), and the imaging device 54c of the macro observation system 54 is The whole observation image of the culture vessel 10 illuminated with the stripe pattern is photographed for each of the set stripe patterns. Therefore, a plurality of image data respectively corresponding to a plurality of types of stripe patterns are stored in the image data storage area of the RAM 63.

  In addition, when the observation set in the observation program is micro observation of the sample at a specific position in the culture vessel 10, the specific position in the culture vessel 10 that has been transported by the transport unit 4 is indicated by the light of the microscopic observation system 55. Position on the axis and place on the sample stage 15, turn on the light source of the second illumination unit 52 or the third illumination unit 53, and cause the imaging device 55c to take a microscopic observation image by transmitted illumination, epi-illumination, and fluorescence. . A signal photographed by the imaging device 55c and inputted to the control unit 6 is processed by the image processing device 100 to generate a microscopic observation image (phase difference image, fluorescent image, etc.), and the image data is an index such as a photographing date / time. Stored in the image data storage area of the RAM 63 together with the data.

  The CPU 61 sequentially executes the above-described whole observation image photographing and microscopic observation image photographing based on the observation schedule set in the observation program. The image data stored in the RAM 63 is read from the RAM 63 in response to an image display command input from the operation panel 71. For example, an entire observation image, a microscopic observation image at a specified time, an analysis result of image analysis, and the like are displayed on the display panel 72. Is displayed.

  In the culture observation system BS configured as described above, when observing the growth state of the fertilized egg J in the culture container 10 (dish 11), the fertilized egg to be observed must be detected from the culture container 10. However, as a series of detection procedures, generally, the culture vessel 10 is photographed by the macro observation system 54 to obtain a whole observation image (macro image), and after detecting the medium drop D from this whole observation image, A flow in which the medium drop D is photographed by the microscopic observation system 55, and the fertilized eggs to be observed injected into the medium drop D one by one (or plural pieces) are distinguished from other foreign substances to recognize the fertilized eggs. It has become. In order to detect a fertilized egg, microscopic observation in a high-magnification visual field is generally required. However, as the observation magnification increases, the observation visual field (observation range) becomes narrower. On the other hand, it is necessary to capture and acquire a large number of images. Therefore, as described above, prior to detection of the fertilized egg by microscopic observation, if the area of the medium drop D containing the fertilized egg can be detected in advance by macro observation, the area for acquiring a high-magnification image in the microscopic observation is also limited thereafter. It becomes possible to shorten the entire observation time.

  However, the color of the medium drop D ranges from red to colorless and transparent, and it is not easy to detect the area of the medium drop D from the tint and brightness density using the entire observation image (macro image). In particular, in the case of the colorless and transparent medium drop D, it is difficult to even visually confirm the area of the medium drop D. Therefore, it is difficult to perform accurate detection only by performing circle detection on a normal whole observation image. It is necessary to first visualize the colorless and transparent medium drop D, and as one of the methods, it is conceivable to place some pattern on the back surface (bottom surface) of the dish 11. Since the pattern placed on the back surface of the dish 11 is modulated by the lens effect of the medium drop D, the medium drop D and the mineral oil O are determined from the difference between the pattern reflected around the medium drop D and the pattern reflected in the medium drop D. The boundary between and can be visually identified. In order to make the observation apparatus perform the same identification as this and detect the medium drop D with high accuracy, it is necessary to model the visual detection and search for a pattern more suitable for detection.

  In the present embodiment, the modulation of the pattern at the boundary portion of the medium drop D is considered as a sudden change in the luminance value due to the break of the pattern, and the detection by the differential filter is performed. A stripe pattern is used as an illumination pattern suitable for detection. When the stripe pattern is used, the change of the pattern inside and outside the medium drop D is as shown in FIG. 6A. Therefore, the region where the luminance differential value in the direction in which the stripe pattern extends is large is the boundary portion of the medium drop D. Can be extracted as Further, if a plurality of whole observation images with the phase of the stripe pattern shifted can be prepared, a differential process is performed on the whole observation image G shown in FIG. As shown in FIG. 6B, it is possible to obtain a composite image GX that can continuously identify the boundary (contour) of the medium drop D over a wide range.

  Detection (identification) of the area of the medium drop D using the composite image GX of the differential image is performed by a predetermined image processing method using the feature that the medium drop D is generally formed in a circular shape. For example, the region of the medium drop D can be detected (identified) by detecting a circle from the composite image GX by Hough transform. In addition, for example, the region of the medium drop D is detected by performing circular extraction by performing integration in the θ direction of the polar coordinates on all points of the composite image GX and extracting points showing peaks in a predetermined radius range. (Identification) may be performed. Further, for example, a region of the medium drop D may be detected (identified) by extracting a circular shape from the composite image GX using a so-called level set method (dynamic contour method). At this time, the size of the image of the medium drop D (for example, about 6 to 7 mm in diameter of about 20 μl) to be imprinted on the image is almost determined by the image acquisition conditions (observation magnification, etc.). The area of an object that can be a candidate (region surrounded by the extracted circular outline) is calculated by a known image processing method, and deviates from the setting range (upper limit value and lower limit value) determined as the discrimination criterion by area Exclude what is.

  Image processing for detecting (identifying) the area of the medium drop D is executed in the image processing apparatus 100 of the culture observation system BS. As illustrated in FIG. 7, the image processing apparatus 100 includes an image storage unit (input unit) 110 that acquires and stores a whole observation image (macro image) obtained by photographing the culture vessel 10 to be observed by the imaging device 54c. An image composition unit 120 that performs differential processing on the entire observation image captured and acquired with a plurality of illumination patterns stored in the image storage unit 110, and generates a composite image by superimposing the differential images, and an image An image analysis unit 130 that detects the medium drop D based on the synthesized image generated by the synthesis unit 120, and a synthesized image generated by the image synthesis unit 120 and a determination result analyzed by the image analysis unit 130 are output to the outside. An output unit 140, and an image and detection result of the medium drop D analyzed by the image synthesis unit 120 and the image analysis unit 130 are displayed on the display panel 7 Configured to output to be displayed on. The image processing apparatus 100 is configured such that an image processing program GP preset and stored in the ROM 62 is read by the CPU 61 and processing based on the image processing program GP is sequentially executed by the CPU 61.

  An image processing method executed by the image processing apparatus 100 configured as described above based on the culture processing image processing program GP will be described with reference to the flowchart shown in FIG. As described above, in the culture observation system BS, observation in the culture vessel 10 designated every predetermined time is performed according to the observation conditions set in the observation program. Specifically, the CPU 61 operates each stage of the transport unit 4 to transport the culture vessel 10 to be observed from the stocker 3 to the observation unit 5 (in this embodiment, disposed on the optical axis of the macro observation system 54). A macro image is photographed by the imaging device 54c of the macro observation system 54.

  The image processing apparatus 100 first acquires a whole observation image (macro image) photographed by the imaging device 54c in step S1, and acquires the acquired image such as the code number, the observation position, and the observation time of the culture vessel 10. It is stored in the image storage unit 110 together with the index data. At this time, a plurality of image data respectively corresponding to a plurality of preset stripe patterns (for example, four types of stripe patterns having different phases by ¼ period) are stored in the image storage unit 110, respectively.

  In step S <b> 2, the image synthesis unit 120 performs differential processing on the entire observation image captured and acquired with a plurality of types of stripe patterns stored in the image storage unit 110, and superimposes and combines the differential images. Generate an image.

  Here, the detailed flow in step S2 will be described with additional reference to FIG. First, the image composition unit 120 acquires a first overall observation image (stripe image) illuminated with the first phase stripe pattern from the image storage unit 110 (step S21a). Next, differential processing is performed on the first overall observation image using a differential filter (step S22a), and a first differential image based on the first overall observation image is generated (step S23a).

  Similarly, a second overall observation image (stripe image) illuminated with a stripe pattern having a second phase different from the first phase is acquired from the image storage unit 110 (step S21b), and the second overall observation image is obtained. Is subjected to differential processing (step S22b), and a second differential image based on the second overall observation image is generated (step S23b). Similarly, a third overall observation image (stripe image) illuminated with a stripe pattern having a third phase different from the first and second phases is acquired from the image storage unit 110 (step S21c), and the third A differential process is performed on the overall observation image (step S22c), and a third differential image based on the third overall observation image is generated (step S23c). Similarly, a fourth overall observation image (stripe image) illuminated with a stripe pattern having a fourth phase different from the first to third phases is acquired from the image storage unit 110 (step S21d). A differential process is performed on the whole observation image (step S22d), and a fourth differential image based on the fourth whole observation image is generated (step S23d).

  When the first to fourth differential images based on the first to fourth overall observation images having different illumination pattern phases are generated, the image composition unit 120 superimposes the first to fourth differential images, respectively ( Step S24), a composite image of each differential image is generated (Step S25). When the composite image is generated in this way, the process proceeds to the next step S3.

  Subsequently, in step S3, the image analysis unit 130 detects (identifies) the region of the medium drop D in the dish 11 from the combined image generated by the image combining unit 120 using the circle detection method described above. . In step S <b> 4, a detection result indicating the region of the medium drop D detected by the image analysis unit 130 is output from the output unit 140.

  The detection result output from the output unit 140 is displayed on the display panel 72 of the operation panel 7, and a contour indicating the region of the medium drop D is displayed in the entire observation image, or a symbol indicating the medium drop D is displayed. For example, “D” is displayed, and the outline of the medium drop D is displayed with a hue and brightness different from those of other regions.

  The detection data output from the output unit 140 is transmitted to a computer or the like externally connected via the communication unit 65 to display a similar image, or the color change (pH change) of the medium drop D or fertilization It can be configured to be used as basic data for observing the growth state of the egg J.

  Thus, the user can immediately recognize the medium drop D by referring to the image displayed on the display panel 72 or the image displayed on a monitor such as an externally connected computer. Only for, a microscopic observation image by the microscopic observation system 55 is acquired (avoid unnecessary shooting of only the background), and the target fertilized egg J can be efficiently detected from the medium drop D.

  Next, a method for producing a fertilized egg as a culture will be outlined with reference to FIG. First, in step S110, a fertilized egg (culture) is injected into the culture vessel 10 (dish 11) together with the medium drop D, and the culture chamber 2 is maintained in an environmental condition suitable for culturing the fertilized egg. The fertilized egg is cultured under the environmental conditions. The environmental conditions are adjusted in the control unit 6 according to the culture environment of the fertilized egg such as the temperature, humidity, and carbon dioxide concentration in the culture chamber 2.

  In step S120, as a pre-stage of observation of the fertilized egg in the culture container 10, the above-described image processing steps S1 to S4 (see FIG. 1) are executed to identify the area of the medium drop D from the dish 11. Next, in step S130, a microscopic observation image (for example, a phase difference image) of the medium drop D is taken by the microscopic observation system 55 using the second illumination unit 52, and fertilization is performed from among a plurality of objects that are imprinted on the observation image. Identify eggs. At this time, one (or more) fertilized eggs are identified for each medium drop D in the culture container 10 (dish 11).

  Subsequently, in step S140, a plurality of fertilized eggs identified for each medium drop D are sorted based on a predetermined sorting criterion. As a selection standard for a fertilized egg, the grade of the fertilized egg is determined based on the timing of cleavage, the shape of the blastomere, and the like, and a good one satisfying this selection standard is selected. For example, it is performed based on whether or not the timing at which cleavage occurs in all egg cells in the egg is in the same period as having passed through a good growth state. That is, regarding the cleavage of normal fertilized eggs, cells of the same generation divide at almost the same timing, and only cells of the same generation exist in the embryo. On the other hand, regarding the cleavage of an abnormal fertilized egg, even when cells are of the same generation, the division timing is shifted, and cells of different generations are mixed in the embryo.

  In step S150, the selected fertilized eggs (good fertilized eggs that have grown to a state called a blastocyst) are collected and stored frozen in, for example, liquid nitrogen at minus 196 ° C. The fertilized egg (blastocyst) is returned to the mother (embryo transfer) at a predetermined cycle. The fertilized eggs to be cultured may be fertilized eggs such as humans, cows, horses, pigs and mice. In addition, the fertilized egg may be stored in a blastocyst state or may be stored in a divided phase (4-cell stage embryo, 8-cell stage embryo).

  As described above, according to the image processing program GP according to the present embodiment, the image processing method and the image processing apparatus 100 configured by executing the image processing program GP, and the culture manufacturing method, the culture medium When identifying the region of the drop D, the medium drop D in the culture vessel 10 (in the dish 11) is based on a plurality of whole observation images having different illumination patterns (stripe patterns) reflected in the culture vessel 10. In order to identify the region, for example, by generating a composite image in which the boundary (outline) of the medium drop D can be continuously identified over a wide range, the medium drop D can be extracted from the culture vessel 10 without depending on the color information. It is possible to accurately identify these areas.

  At this time, a plurality of differential images obtained by performing differential processing on the plurality of overall observation images are superimposed to generate a composite image of the differential image, and by detecting a circle from the composite image, the culture vessel If the area of the medium drop D in 10 (in the dish 11) is identified, it is possible to reliably generate a composite image that can continuously identify the boundary (contour) of the medium drop D over a wide range, It becomes possible to identify the region of the medium drop D from the culture container 10 with higher accuracy.

  Moreover, the image processing calculation for identifying the area | region of the culture medium drop D can be made easy by using the stripe pattern which has a periodic striped pattern as an illumination pattern.

  In the above-described embodiment, and at this time, the form of the illumination pattern (stripe pattern) reflected in the culture vessel 10 is obtained by changing the phase of the stripe pattern by the first illumination unit 51 that is a transmissive liquid crystal panel. A plurality of different overall observation images are acquired, but the present invention is not limited to this. For example, by moving the culture vessel 10 (dish 11) in the optical axis direction in the pan-focus imaging system, an illumination pattern that is reflected in the culture vessel 10 using the lens effect (change in the composite magnification) of the medium drop D You may make it acquire the several whole observation image which changed this.

  Accordingly, a modification of the first illumination unit will be described with reference to FIG. The first illuminating unit 151 according to the modification includes a transmissive liquid crystal panel 152 with a backlight provided on the lower frame 1b side and a culture vessel 10 (dish 11) positioned on the transmissive liquid crystal panel 152 as a macro observation system. 54, and an arm mechanism (not shown) that moves in the optical axis direction. The entire culture vessel 10 is transmitted and illuminated from below the sample stage 15 by the transmission type liquid crystal panel 152. The transmissive liquid crystal panel 152 has the same configuration as that of the first illumination unit 51 described in the above embodiment, and detailed description thereof is omitted. However, a binary pattern (a light pattern and a dark part are repeatedly arranged as illumination patterns) Stripe pattern). In addition, not only the transmissive liquid crystal panel 152 but also a configuration in which a pattern forming member for forming a stripe pattern as an illumination pattern is disposed on the upper surface of a surface light source such as an LED.

  The arm mechanism (not shown) includes a lower position where the bottom of the culture vessel 10 (dish 11) is in contact with the transmissive liquid crystal panel 152 as shown in FIG. 10A, and a culture as shown in FIG. 10B. The culture vessel 10 can be moved up and down along the optical axis of the macro observation system 54 between the bottom of the vessel 10 and an upper position away from the transmissive liquid crystal panel 152. The moving range of the culture vessel 10 is preferably set in a range where a virtual image is generated behind the background pattern from a range where the background pattern is primarily imaged within the forward depth.

  By the way, when the illumination light from the transmissive liquid crystal panel 152 is transmitted through the culture vessel 10 (dish 11), the refractive index of the mineral oil O is usually about 1.44 and the refractive index of the medium drop D is about 1.33. Medium drop D acts as a negative lens. Therefore, when the culture vessel 10 (dish 11) is moved from the lower position to the upper position by the arm mechanism (not shown), the magnification of the macro observation system 54 is increased or decreased by the lens effect of the medium drop D. Therefore, the form (period, phase, etc.) of the stripe pattern reflected in the culture vessel 10 can be changed. In FIG. 10, only the principal ray is shown for simplicity, and the refraction of the mineral oil O and the medium drop D is ignored.

  Using the first illumination unit 151 configured as described above, the culture vessel 10 illuminated on the transmissive liquid crystal panel 152 is moved from the lower position to the upper position along the optical axis of the macro observation system 54. While the distance between the medium drop D (dish 11) and the transmissive liquid crystal panel 152 is changed, the whole observation image of the culture vessel 10 is photographed at a plurality of height positions by the imaging device 54c of the macro observation system 54. For example, it is possible to acquire a plurality of whole observation images having different illumination patterns (stripe patterns) reflected in the culture vessel 10. Note that the size of the medium drop D or the like that appears in the entire observation image changes as the culture vessel 10 moves, but it can be corrected based on the amount of movement of the culture vessel 10 or the magnification of the macro observation system 54.

  As a result, by applying the same image processing method as in the above-described embodiment, it is possible to accurately identify the area of the medium drop D from the culture container 10 without depending on the color information. In this case, in step S2, differential processing is performed on each of the plurality of whole observation images photographed and acquired at a plurality of height positions, and each differential image is superimposed to generate a composite image. Thereby, in step S3, the area | region of the culture medium drop D in the dish 11 can be similarly detected from the synthetic | combination image produced | generated by step S2 using the method of a circle detection mentioned above.

  In the above-described modification, a plurality of differential images obtained by performing differential processing on a plurality of overall observation images are superimposed to generate a composite image of the differential image, and a circle is detected from the composite image. Thus, the region of the medium drop D in the culture vessel 10 (in the dish 11) is identified, but the present invention is not limited to this. For example, based on the difference between the whole observation images, a change in the stripe pattern reflected in the culture vessel 10 is obtained by optical flow, and a portion where the pattern changes in the whole observation image is detected (identified). ). For example, a known gradient method or block matching method can be used as a method for estimating a change in pattern by optical flow.

  In the above-described modification, when a telecentric optical system is used as the macro observation system 54, the image of the culture vessel 10 does not change even if the culture vessel 10 is moved within the forward depth. It is possible to remove the writing in the dish 11 by taking the difference between the images, and at this time, the writing drop can be removed only by a simple difference process to extract the region of the medium drop D. More preferable.

  In the above-described modification, the culture vessel 10 (dish 11) is moved in the optical axis direction of the macro observation system 54 with respect to the transmissive liquid crystal panel 152 (stripe pattern). However, the present invention is not limited to this. The transmissive liquid crystal panel 152 (stripe pattern) may be moved in the optical axis direction of the macro observation system 54 with respect to the culture vessel 10 (dish 11). In this case, since the relative positional relationship between the dish 11 and the macro observation system 54 does not change, even if writing is performed on the upper surface or the lower surface of the dish 11, the writing is removed by taking the difference between the entire observation images. Therefore, the detection efficiency of the medium drop D is not changed.

  In the above-described embodiment, the stripe pattern is used as the illumination pattern. However, the present invention is not limited to this, and a lattice pattern may be used.

BS culture observation system GP image processing program D medium drop J fertilized egg 5 observation unit 6 control unit 7 operation panel 10 culture vessel 11 dish 54 macro observation system 54c imaging device 61 CPU 62 ROM
63 RAM
DESCRIPTION OF SYMBOLS 100 Image processing apparatus 110 Image memory | storage part (input part) 120 Image composition part 130 Image analysis part 140 Output part

Claims (12)

Transmitting and illuminating the inside of the culture vessel containing the medium used for culturing the culture with the illumination light having a predetermined illumination pattern, and obtaining an observation image obtained by photographing the inside of the culture vessel that has been transmitted and illuminated with an imaging device ,
Based on the observation image in which the illumination pattern is reflected in the culture container, the medium in the culture container is identified,
When obtaining the observation image, obtain a plurality of observation images having different forms of the illumination pattern reflected in the culture vessel,
A culture observation image processing method characterized in that, when identifying the culture medium, the culture medium in the culture vessel is identified based on the plurality of observation images.   When identifying the culture medium, a plurality of differential images obtained by differentiating each of the plurality of observation images are superimposed to generate a composite image of the differential image, and a circle is detected from the composite image. The culture processing image processing method according to claim 1, wherein the culture medium in the culture container is identified.   The culture processing image processing method according to claim 1, wherein the illumination pattern has a periodic stripe pattern. An image processing program that allows a computer to function as an image processing apparatus that can be read by a computer and that acquires an image captured by an imaging apparatus and performs image processing.
By illuminating the inside of the culture container containing the medium used for culture culture with illumination light having a predetermined illumination pattern, an observation image obtained by photographing the inside of the transmissively illuminated culture container with an imaging device is obtained. Steps,
Identifying the medium in the culture vessel based on the observation image in which the illumination pattern is reflected in the culture vessel;
Outputting the identification result of the medium;
Realizing the computer,
In the step of obtaining the observation image, obtaining a plurality of observation images having different forms of the illumination pattern reflected in the culture vessel,
A culture observation image processing program characterized in that, in the step of identifying the culture medium, the culture medium in the culture vessel is identified based on the plurality of observation images.   In the step of identifying the culture medium, generating a composite image of the differential image by superimposing a plurality of differential images obtained by differentiating the plurality of observation images, and detecting a circle from the composite image 5. The culture observation image processing program according to claim 4, wherein the culture medium in the culture container is identified.   6. The culture observation image processing program according to claim 4, wherein the illumination pattern has a periodic stripe pattern. By illuminating the inside of the culture container containing the medium used for culture culture with illumination light having a predetermined illumination pattern, an observation image obtained by photographing the inside of the transmissively illuminated culture container with an imaging device is obtained. An input section;
An image analysis unit for identifying the culture medium in the culture vessel based on the observation image in which the illumination pattern is reflected in the culture vessel acquired by the input unit;
An output unit that outputs the identification result of the medium by the image analysis unit,
The input unit acquires a plurality of observation images having different forms of the illumination pattern reflected in the culture vessel,
The image processing unit for culture observation, wherein the image analysis unit identifies the culture medium in the culture vessel based on the plurality of observation images.   The image analysis unit generates a composite image of the differential image by superimposing a plurality of differential images obtained by performing differential processing on the plurality of observation images, and detects a circle from the composite image, The culture processing image processing apparatus according to claim 7, wherein the culture medium in the culture container is identified.   The image processing apparatus for culture observation according to claim 7 or 8, wherein the illumination pattern has a periodic stripe pattern. Culturing the culture under the specified environmental conditions,
The culture medium, wherein the culture medium is identified from the culture container in which the culture medium in which the culture is cultured is stored using the image processing apparatus according to any one of claims 7 to 9. Method. Culturing the culture under the specified environmental conditions,
The inside of the culture container in which the culture medium in which the culture is cultured is stored is transmitted and illuminated with illumination light having a predetermined illumination pattern, and an observation image obtained by imaging the inside of the transmitted culture container with an imaging device is obtained. ,
Based on the observation image in which the illumination pattern is reflected in the culture container, the medium in the culture container is identified,
When obtaining the observation image, obtain a plurality of observation images having different forms of the illumination pattern reflected in the culture vessel,
When identifying the culture medium, the culture medium in the culture container is identified based on the plurality of observation images. Selecting a culture contained in the identified medium based on a predetermined selection criterion;
The method for producing a culture according to claim 10 or 11, wherein the selected culture is collected from the culture vessel and stored.

Images