JP2016090234A - Image processing device, image processing program, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device and the like allowing a user to accurately recognize a state of an object cell while recognizing relevance of a plurality of cell images in a target cell desired by the user.SOLUTION: An image processing device 30 processes a plurality of slice images associated with at least any one of a height-directional distance of a fertilized egg and a time when the fertilized egg is photographed. The image processing device includes: an image designation section 321 for designating a specific slice image out of a plurality of slice images; an image selection section 322 for selecting at least one slice image different from the specific slice image out of the multiple slice images, as a selection slice image; a feature amount extraction section 323 for extracting a feature amount from the selected at least one selection slice image; and a display control section 350 causing a display part 340 to display the feature amount image including information on the extracted feature amount as overlaid upon the specific slice image.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing apparatus, an image processing program, and an image processing method used for observing cultured cells, for example.

  In recent years, research has progressed in the field of cell culture including regenerative medicine and infertility treatment. At the same time, there is a technology for easily observing the culture state of cells collected from living bodies without affecting the cells. It is requested.

  Conventionally, as this type of apparatus, it is known to use an image processing apparatus because cells can be observed non-invasively. Particularly in cell observation, in many cases, the position (height and time) of the cell to be observed in the Z-axis direction and the time-axis direction is generally determined. There is known a technique in which slice images acquired from a specific region to be observed by stack photographing are captured and the slice images are displayed in a matrix with a Z axis and a time axis (for example, Patent Document 1).

JP 2008-139579 A

  However, the apparatus described in Patent Document 1 only has a configuration in which a large number of slice images are displayed in a matrix on the screen, and allows the user to recognize the relevance of a plurality of cell images desired by the user. However, it is difficult to accurately check the state of the cells.

  The present invention has been made to solve the above-described problems, and its purpose is to accurately recognize the state of the target cell while recognizing the relevance of a plurality of cell images in the target cell desired by the user. An object is to provide an image processing apparatus or the like that can be confirmed.

  In order to solve the above-described problem, the present invention provides an image processing for processing a plurality of cell images in a target cell to be observed associated with at least one of a distance in a specific direction of the cell and a time at which the cell is imaged. In the apparatus, a specific cell image designating unit for designating a specific cell image from the plurality of cell images, and a cell for selecting at least one cell image different from the specific cell image as a selected cell image from the plurality of cell images Image selection means; feature quantity extraction means for extracting feature quantities from the selected at least one selected cell image; and display means for superimposing a feature quantity image including information on the extracted feature quantities on the specific cell image And a display control means for displaying on the display.

  With this configuration, since the display control unit displays the feature amount image including the feature amount information of the selected cell image on the display unit on the display unit, the display control unit displays the specific cell image and the selected cell desired by the user. In addition to recognizing the relationship with the image, the state of the specific cell image and the selected cell image can be confirmed accurately.

  The image processing apparatus of the present invention can accurately check the state of the target cell while recognizing the relevance of a plurality of cell images in the target cell desired by the user.

It is a block diagram which shows the structure of one Embodiment of the cell observation assistance system which concerns on this invention. It is a block diagram which shows the structure of the imaging device in the cell observation assistance system of one Embodiment. It is a block diagram which shows the structure of the image processing apparatus in the cell observation assistance system of one Embodiment. It is a figure which shows an example of the data of the slice image recorded on the image data recording part in the image processing apparatus of one Embodiment. It is a figure which shows the state by which the slice image in one Embodiment was imaged with the marker. It is a figure for demonstrating the function of each part of the data processing part in the image processing apparatus of one Embodiment. It is explanatory drawing regarding extraction of the feature-value of the feature-value extraction part in the image processing apparatus of one Embodiment. It is a flowchart which shows the operation | movement of the cell observation assistance process performed in the imaging device of one Embodiment. It is a flowchart which shows the operation | movement of the cell observation assistance process performed in the image processing apparatus of one Embodiment. It is an example of the data of the slice image for confirming how the state of a fertilized egg is changing at the same height in the modification of one Embodiment. It is a figure which shows an example at the time of reading the data of the slice image with the same fertilized egg ID from the image data recording part in the modification of one Embodiment. It is an example of the culture container used for the cell observation assistance system in the modification of one Embodiment. It is a figure which shows the example of identification of the well of the culture container used for the cell observation assistance system in the modification of one Embodiment.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In the following embodiments, the image processing apparatus and the image processing according to the present invention are applied to a cell observation support system that supports the observation of a fertilized egg by displaying a slice image of a cultured fertilized egg (embryo) to a user. It is an embodiment when a program and an image processing method are applied. However, the present invention is not limited to the following embodiments as long as the technical idea is included.

[1] Outline of Image Processing Apparatus First, the configuration and outline of a cell observation support system 1 in the present embodiment will be described with reference to FIG. FIG. 1 is a system configuration diagram showing the configuration of the cell observation support system 1 in the present embodiment.

  The cell observation support system 1 according to the present embodiment, for example, applies a target cell to be observed placed in a predetermined container such as a dish along the Z-axis direction perpendicular to the XY plane on which the container is arranged, every predetermined time. In this system, the slice image captured while changing the distance from a predetermined reference position at a predetermined pitch is displayed to the user to support cell observation. The Z-axis direction corresponds to a specific direction according to the present invention and is referred to as a “height direction” in the following description.

  For example, the cell to be observed includes a fertilized egg used for infertility treatment. In fertility treatment, generally, after collecting an egg and performing in vitro fertilization, the obtained fertilized egg is cultured in a predetermined container, and then developed to a suitable developmental stage and transplanted into the uterus. Has been done.

  In particular, after fertilization, two pronuclei that are the nuclei of the egg and sperm are formed in the ovum, and after the two pronuclei are fused, cell division occurs and cleavage occurs. Also, two days after fertilization, the fertilized egg goes through the 2-cell stage to the 4-cell stage, and 3 days after fertilization, the fertilized egg goes into the 8-cell stage. Furthermore, 4 days after fertilization, cell division further proceeds and each cell boundary becomes a morula that contacts each other, and 5 days after fertilization, the egg cell further divides into a blastocyst . And this blastocyst is comprised from the trophectoderm and the inner cell mass in the inside. In addition, there is an error of around several hours in each occurrence of cleavage, and observations that take this into account are necessary.

  On the other hand, in the process of transplanting fertilized eggs into the mother body for infertility treatment, in order to ensure reliable implantation and subsequent growth, the fertilized eggs are cultured for 2 to 5 days after fertilization, and then suitable for transplantation. Only fertilized eggs in good stages are selected and transplanted.

  Therefore, in order to observe the culture state, a large number of slice images obtained by imaging the fertilized egg while changing the height dimension at a predetermined pitch are obtained every predetermined time, and the state of the developmental stage of the fertilized egg is thereby observed. Yes.

  In many cases of observation of a fertilized egg, the position to be observed (height and time) in the height direction and time axis direction of the fertilized egg is generally determined, so that the image is captured at a specific height and time. A slice image (hereinafter referred to as “specific slice image”) and a related slice image, for example, a slice image spatially adjacent to a specific slice image in the height direction or a slice image adjacent in time can be confirmed. Observation work becomes easier.

  Therefore, in the present embodiment, a feature image of a slice image spatially adjacent to a specific slice image specified by, for example, a user from among a large number of captured slice images in the height direction with respect to the specific slice image. Alternatively, the feature amount images of slice images that are temporally adjacent to each other are displayed in an overlapping manner, and the state of the target cell is accurately confirmed while recognizing the relevance of a plurality of cell images in the target cell desired by the user. Can be done.

  Specifically, as shown in FIG. 1, the cell observation support system 1 of the present embodiment acquires a plurality of slice images obtained by imaging a fertilized egg while changing the height dimension at a predetermined pitch every predetermined time. The imaging device 10 capable of performing image processing, the network 20, and an image processing device 30 that performs predetermined image processing on a specific slice image specified by a user among a plurality of slice images, for example.

  The imaging apparatus 10 is an apparatus having a communication function for connecting to the network 20 to exchange data, an imaging function capable of acquiring a slice image, and a microscope function for observing a fertilized egg, for example.

  In particular, the imaging device 10 uses the imaging function to form a still image of the fertilized egg to be observed that is placed in the dish well while changing the height dimension at regular intervals (for example, every 6 minutes or 12 minutes). The slice image data is generated, and the generated slice image data is transmitted to the image processing device 30 together with the time information indicating the height at the time of imaging and the imaging time.

  For example, the imaging apparatus 10 includes an optical system, a CCDI sensor (Charge Coupled Device Image Sensor) that converts an optical image input from the optical system into an electrical signal, and image data based on the electrical signal generated by the CCDI sensor. And a generating unit that generates

  Further, when transmitting the image data of the slice image to the image processing device 30, the imaging device 10 uses a predetermined communication standard such as a LAN (Local Area Network) or the like directly or via an access point (not shown). It has the structure which transmits to the image processing apparatus 30 by radio | wireless.

  The network 20 is constituted by, for example, a wired or wireless IP (Internet Protocol) network, or a public telephone line network including a mobile phone network or other networks.

  The image processing apparatus 30 is an image processing apparatus that cooperates with the imaging apparatus 10 to display to the user an image obtained by performing predetermined image processing on the slice image data generated by the imaging apparatus 10 and support cell observation. is there.

Specifically, the image processing apparatus 30
(1) Acquire and record a plurality of slice images including height and imaging time information at the time of imaging from the imaging device 10,
(2) A specific slice image is specified from a plurality of recorded slice images,
(3) Select a slice image (hereinafter referred to as “selected slice image”) that is related to the specific slice image and is selected according to a predetermined condition,
(4) extracting the feature amount of the selected slice image;
(5) generating a feature amount image including the extracted feature amount information;
(6) The generated feature amount image is combined with the specific slice image and displayed.
Thus, a process for supporting cell observation for the user (hereinafter referred to as “cell observation support process”) is executed.

  In particular, in the present embodiment, the image processing apparatus 30 has a configuration in which a slice image spatially adjacent to a specific slice image in the height direction or a slice image temporally adjacent to the specific slice image is used as the selected slice image. ing.

  With such a configuration, the cell observation support system 1 of the present embodiment can accurately confirm the state of the target cell while recognizing the relevance of a plurality of cell images in the target cell desired by the user. It has become.

[2] Imaging Device Next, the configuration of the imaging device 10 of the present embodiment will be described using FIG. FIG. 2 is a block diagram illustrating a configuration of the imaging apparatus 10 according to the present embodiment.

  As shown in FIG. 2, the imaging apparatus 10 images a stage drive unit 102 that drives a stage 101 that can move along the XYZ axes, and a fertilized egg in a container disposed on the stage 101 along the Z-axis direction. The image capturing unit 103 acquires slice images, and the group ID assigning unit 104 assigns IDs, which are identification information for identifying each group when the captured images are grouped, to each group.

  In addition, the imaging apparatus 10 includes an operation unit 105 that is operated by a user to input information related to an imaging operation, and a communication control unit 106 that exchanges various data such as slice image data transmitted to the image processing apparatus 30. A management control unit 107 that controls each unit, and a bus 11 that transfers data between the components.

  The imaging device 10 may include a display unit that monitors and displays a fertilized egg to be imaged, a display control unit that controls the display unit, and a recording unit that temporarily records slice image data. Good.

[3] Image Processing Device Next, the configuration of the image processing device 30 in this embodiment will be described with reference to FIGS. 3 and 4.

  FIG. 3 is a block diagram illustrating the configuration of the image processing apparatus 30 according to the present embodiment, and FIG. 4 illustrates slice image data recorded in the image data recording unit 302, which is grouped for each imaging time. An example is shown.

  In particular, FIG. 4 shows an image (that is, a slice image) obtained by sequentially changing the height at the time of imaging of the 4-cell stage embryo in the 4-cell stage from the image names A1 to A5 in steps of 50 μm. It is the figure which showed typically four cells of an embryonic stage, ie, four blastomeres.

  As illustrated in FIG. 3, the image processing apparatus 30 according to the present embodiment includes a data recording unit 300 that records various data used when various programs are executed, and a slice image transmitted from the imaging apparatus 10. It has a communication control unit 310 that exchanges various data such as data, and a data processing unit 320 that executes cell observation support processing from the acquired slice image.

  The image processing apparatus 30 according to the present embodiment includes a display unit 340 configured by a liquid crystal panel, a display control unit 350 that controls the display unit 340, an operation unit 370, and a management control unit 380 that controls each unit. ,have.

  Note that the above-described units are connected to each other via a bus 31, and data transfer is performed between the components.

  The data recording unit 300 is configured by, for example, an HDD (Hard Disc Drive) or the like, and is generated by being imaged by the application recording unit 301 in which an application program for executing each process such as a cell observation support process is recorded, and the imaging apparatus 10. And an image data recording unit 302 for recording slice image data, and a ROM / RAM 303 used as a work area during execution of each program.

  In particular, in the image data recording unit 302, a plurality of slice images acquired from the imaging device 10 are classified and recorded for each predetermined group together with information on the height and imaging time at the time of imaging. Each slice image is recorded for each group, for example, every imaging time, every height, every dish, or every well.

For example, in the image data recording unit 302 of the present embodiment, as shown in FIG.
(1) Image name,
(2) Same time group ID,
(3) Group imaging time,
(4) Z-axis height, and
(5) Slice images associated with individual image capturing times are recorded.

  Specifically, the image name is set, for example, by the user operating the operation unit 370 or automatically in the imaging device 10, and the same time group ID is grouped for each imaging time. This is used in order to determine the group imaging time.

  The Z-axis height indicates an imaging position from a predetermined reference position in the Z-axis direction in the imaging apparatus 10, and the individual image imaging time indicates an individual imaging time when each slice image is captured. ing.

  The predetermined reference position in the Z-axis direction is an arbitrary position set by the user in the Z-axis direction. For example, for positioning of the well bottom surface, the dish bottom surface, the upper vertex of the fertilized egg, and the slice image The upper part of the marker 40 (after-mentioned) to be used can be mentioned.

  FIG. 4 shows that the five slice images with the image names A1 to A5 belong to the same group associated with the imaging time, that is, the same group imaging time “001”. The individual imaging time “2014/07/05 09:25:00” of the first imaged image name A1 is shown.

  FIG. 4 shows that the slice image with the image name A1 is captured at a position 50 μm in height from the reference position, and one of the four blastomers has a height of 100 μm (with the image name A2). In the slice image, two images are captured at a height of 150 μm (slice image having an image name A3), and the other one is captured at a height of 200 μm (slice image having an image name A4).

  The ROM / RAM 303 stores various programs necessary for driving the image processing apparatus 30. In particular, various applications executed by the data processing unit 320 and the management control unit 380 are recorded in the ROM / RAM 303. The ROM / RAM 303 is used as a work area during execution of each program.

  The communication control unit 310 is a predetermined network interface, establishes a communication line with the imaging device 10, and exchanges various data acquired by the imaging device 10.

The data processing unit 320 is based on an application that executes the cell observation support process recorded in the ROM / RAM 303.
(1) A process of designating a specific slice image from a plurality of slice images acquired from the imaging device 10 (hereinafter referred to as “specific image designating process”),
(2) A process of selecting at least one selected slice image that is related to the specific slice image and is different from the specific slice image (hereinafter referred to as “image selection process”),
(3) Processing for extracting feature amounts from the selected slice image (hereinafter referred to as “feature amount extraction processing”),
(4) Processing for superimposing a feature amount image including extracted feature amount information on a specific slice image (hereinafter referred to as “image composition processing”),
Execute.

  In particular, the data processing unit 320 executes an application to execute an image designation unit 321 that executes a specific image designation process, an image selection unit 322 that executes an image selection process, and a feature quantity extraction process that performs a feature quantity extraction process. A unit 323 and an image composition unit 324 that executes image composition processing are realized.

  For example, the image designation unit 321 of the present embodiment constitutes a specific cell image designation unit of the present invention, and the image selection unit 322 constitutes a cell image selection unit of the present invention. For example, the feature quantity extraction unit 323 of the present embodiment constitutes a feature quantity extraction unit of the present invention.

  Details of each part of the data processing unit 320 in this embodiment will be described later.

  The display unit 340 is configured by a panel of a liquid crystal element or an EL (Electro Luminescence) element, and displays a predetermined image based on display data generated by the display control unit 350. For example, the display unit 340 of the present embodiment constitutes a display unit of the present invention.

  The display control unit 350 generates drawing data necessary for causing the display unit 340 to draw a predetermined image under the control of the management control unit 380 and the data processing unit 320, and generates the generated drawing data on the display unit 340. Output. For example, the display control unit 350 of the present embodiment constitutes the display control means of the present invention.

  The operation unit 370 includes various buttons, operation buttons for inputting operation commands, and a number of keys such as a numeric keypad. The operation unit 370 includes a touch sensor provided on the display unit 340 and is used when performing each operation. It is like that.

  The management control unit 380 is mainly configured by a central processing unit (CPU), and performs overall management control of the image processing apparatus 30 and control of various other processes by executing a program.

[4] Slice Image Next, the slice image data recorded in the image data recording unit 302 will be described with reference to FIG. FIG. 5 is a diagram illustrating a state in which the slice image according to the present embodiment is captured together with the marker.

  Each slice image is an image obtained by imaging a 4-cell stage embryo at a distance indicated by the Z-axis height from the reference position. As shown in FIG. 5, this slice image is taken together with the marker 40. The marker 40 performs positioning on the XY plane and constitutes a positioning portion of the present invention, and is formed, for example, on a container. However, this marker is not shown in FIG. 4 described above.

  In particular, the imaging device 10 according to the present embodiment captures an image so that the entire 4-cell stage embryo and at least a part of the marker 40 are within the imaging range at the time of imaging, and the image processing device 30 performs alignment of slice images. A marker 40 is used. Therefore, the image processing apparatus 30 can perform alignment of each slice image with reference to the marker 40 even if the alignment of XY is slightly shifted during imaging.

  Actually, the shape of the marker 40 is blurred according to the height at the time of imaging, and a difference occurs in how the marker 40 is captured. However, the image processing apparatus 30 according to the present embodiment aligns the shape of the captured marker 40 itself by autocorrelation, for example, without correcting the difference in image appearance. However, the image processing device 30 corrects the position of the marker 40 by, for example, cutting the blurred portion with an image filter or obtaining the center of gravity position of the blurred portion in order to eliminate the difference in how the marker 40 is captured. Then, alignment may be performed on the corrected result by autocorrelation.

[5] Data Processing Unit Next, details of the data processing unit 320 in the image processing apparatus 30 according to the present embodiment will be described with reference to FIGS. 6 and 7. FIG. 6 is a diagram for explaining the function of each unit of the data processing unit 320 according to the present embodiment, and is an explanatory diagram regarding feature amount extraction by the feature amount extraction unit according to the present embodiment.

  The image designation unit 321 acquires information on a group of slice images designated by the user by operating the operation unit 370 from the operation unit 370, designates a plurality of slice images belonging to the obtained group, and sets the data of the images as images. A process of reading from the data recording unit 302 is executed.

  For example, in this embodiment, the image designation unit 321 designates the same time group ID = 001 (hereinafter referred to as “designated group”) shown in FIG. A simplified representation of the data of a plurality of slice images belonging to this designated group is shown in FIG. However, in FIG. 6A, for the sake of explanation, the data of the image name, the Z-axis height, and the slice image are extracted from the data of the designated group shown in FIG.

  On the other hand, the image designation unit 321 acquires information on a specific slice image designated by a user operation from a plurality of slice images belonging to the designated group from the operation unit 370, and designates the obtained specific slice image. Execute the process.

  For example, the image designation unit 321 of the present embodiment designates the slice image with the image name A3 shown in FIG. 4 as the specific slice image A3. That is, in this case, as shown in FIG. 6B, the image designating unit 321 includes a 4-cell stage embryo 51 and two blastomeres 51a and 51b among the four blastomeres contained therein. , A specific slice image A3 is designated.

  The image selection unit 322 executes image selection processing that selects at least one selected slice image that is a slice image related to the specific slice image A3 and is different from the specific slice image A3.

  For example, when the specific slice image A3 is designated, the image selection unit 322 of the present embodiment selects two slice images spatially adjacent to the specific slice image A3 in the height direction. That is, as shown in FIG. 6C, the image selection unit 322 uses two slices of an image name A2 and an image name A4 as two slice images spatially adjacent to the specific slice image A3 in the height direction. Images are selected as selected slice images A2 and A4.

  In particular, the selected slice images A2 and A4 are related slice images in that they belong to the same group as the specific slice image A3.

  In addition, the slice images shown in FIG. 6A are sequentially picked up in steps of 50 μm in height, and the specific slice image A3 is picked up immediately after (just after) the selected slice image A2. And is taken immediately before (immediately before) the selected slice image A4. Therefore, the two selected slice images A2 and A4 are slice images spatially adjacent to the specific slice image A3 in the height direction.

  In this specification, for example, with respect to the specific slice image A3, the selected slice image A2 is referred to as a spatially previous image, and the selected slice image A4 is spatially captured one after. That's it.

  The feature amount extraction unit 323 executes feature amount extraction processing for extracting a feature amount from the selected slice image. Specifically, the feature amount extraction unit 323 extracts the feature amount of the selected slice image in order to obtain information that is considered important for the evaluation of a fertilized egg, such as the number of blastomeres, the number of nuclei, and fragments by cleavage. The extracted feature amount is imaged.

  For example, when the selected slice images A2 and A4 are selected by the image selection unit 322, the feature amount extraction unit 323, as shown in FIG. 6C, features one blastomere feature amount from the selected slice image A2. An image 52 is generated, and a feature image 53 of one blast ball is generated from the selected slice image A4.

In particular, in the present embodiment, the feature amount extraction unit 323 is, for example,
(1) Edge information (2) Luminance information (3) Texture information (4) A feature amount of a selected slice image such as object recognition is extracted, and a feature amount image including the feature amount information is acquired.

(1) Edge information Since the edges of individual blastomeres that appear during repeated cleavage tend to have strong edges, a feature image can be obtained by extracting regions with strong edges as feature values. It has become. For example, as shown in FIG. 7A, the boundary 61a of the blast ball 61 has a feature that an edge appears strongly and a feature that a black image is taken because the luminance is low. FIG. 7A shows a blast ball 61 generated by cleavage.

  Therefore, the feature amount extraction unit 323 according to the present embodiment detects (extracts) edges by various detection methods such as the well-known Canny method, the zero-crossing method, and the Sobel method. In order to remove noise, the feature amount extraction unit 323 according to the present embodiment may connect edge regions and exclude small-sized regions.

(2) Luminance information Boundaries of individual blastomeres that appear during repeated cleavage and portions where fragments occur are extracted with low luminance values because the luminance is often low and black. Etc. are considered. For example, as shown in FIG. 7B, the fragment 62 has a feature that it is captured in black because of its low brightness, and has unique texture information. FIG. 7B shows a fragment 62 that is generated during repeated cleavage.

  Therefore, the feature amount extraction unit 323 of the present embodiment extracts a region having a low luminance value as a feature amount using the threshold value information of luminance. Fragments are cytoplasmic fragments such as crushed bubbles, and embryos with many fragments are said to be difficult to implant, so they are important as one of the features. .

(3) Texture information Although the core has few edges, it has unique texture information such as many fine irregularities and different brightness distribution in the local area compared to other parts. For example, as shown in FIG. 7C, the pronucleus 63 has a feature of high luminance dispersion and a feature of having unique texture information. FIG. 7C shows the pronucleus 63.

  Therefore, the feature amount extraction unit 323 according to the present embodiment detects and extracts a region using a feature amount representing a texture such as luminance dispersion in a local region, a response vector of a filter bank, and texton (Texton).

(4) Object recognition A target object can be recognized by further performing a predetermined process on the basic information amount obtained as described above.

  For example, the feature amount extraction unit 323 according to the present embodiment recognizes a blast ball by performing circle detection after edge extraction, and extracts only edge information representing a circle as a feature amount.

(5) Others The feature quantity extraction unit 323 of the present embodiment may arbitrarily combine the above-described methods, that is, extract a plurality of feature quantities.

  In addition, the feature amount extraction unit 323 according to the present embodiment may extract only information to be extracted by using a plurality of feature amount images obtained by the above method and using the difference. For example, when the brightness information includes information on the blast ball and the fragment, the feature amount extraction unit 323 cancels (subtracts) the information on the blast ball extracted from the brightness information using the edge information, and information on the fragment May be extracted, or other information may be used depending on the information desired to be included in the feature amount image.

  When extracting the feature amount image, the feature amount extraction unit 323 creates a high-dimensional feature amount vector using a plurality of feature amounts, and uses, for example, a statistical method such as principal component analysis to calculate the feature amount image. Or an object may be detected using a machine learning method.

  The image composition unit 324 executes image composition processing in which a feature amount image including feature amount information extracted by the feature amount extraction unit 323 is superimposed on the specific slice image A3. For example, when the specific slice image shown in FIG. 6B is A3 and the selected slice images shown in FIG. 6C are A2 and A4, the image composition unit 324 is as shown in FIG. Then, the composite image 50 is generated by executing image composition processing for superimposing the feature amount images 52 and 53 including the feature amount information extracted by the feature amount extraction unit 323 on the specific slice image A3.

  In particular, the image composition unit 324 includes the images of the 4-cell stage embryo 51, the blastomeres 51a and 51b included in the specific slice image A3, and the selected slice image A2 spatially adjacent to the specific slice image A3 in the height direction. A composite image 50 including the blast ball feature amount images 52 and 53 respectively obtained from A4 is generated.

  Then, the image composition unit 324 outputs the generated image data of the composite image 50 to the display control unit 350 via the bus 31.

  The display control unit 350 generates drawing data necessary for causing the display unit 340 to draw a predetermined image, and displays the generated drawing data on the display unit 340.

  Since the composite image 50 can be displayed on the display unit 340 as described above, the specific slice image A3, and the selected slice images A2 and A4 captured before and after the specific slice image A3, respectively. And the state of the blastomeres 51a and 51b in the specific slice image A3 and the state of the blastomeres in the selected slice images A2 and A4 can be confirmed accurately by the user. .

[6] Operation of Cell Observation Support System [6.1] Operation of Imaging Device Next, the operation of the cell observation support process executed in the imaging device 10 of the present embodiment will be described using FIG. FIG. 8 is a flowchart showing the operation of the cell observation support process executed in the imaging apparatus 10.

  In this operation, the imaging apparatus 10 is turned on, the reference position in the Z-axis direction is set to 0 μm, and the fertilized egg to be imaged in the container placed on the stage 101 is captured by the marker 40 (see FIG. 5). It is assumed that it is set so as to fall within the range, and is executed after communication with the image processing apparatus 30 is established.

  In addition, this operation is a stage 101 height that is adjusted step by step each time an image is taken (hereinafter referred to as “imaging height”, the height for determining the above Z-axis height) and It is assumed that the number n of imaging based on the predetermined number is set in advance automatically or based on a user operation.

  First, when the management control unit 107 detects a user operation input via the operation unit 105 (step S100), the imaging unit 103 initializes a variable n indicating the number of times of imaging (step S101), and the value of n Is incremented (step S102).

  Next, the stage driving unit 102 sets the stage 101 to a predetermined imaging height based on the value of n under the control of the management control unit 107 (step S103), and the imaging unit 103 is controlled by the management control unit 107. The n-th fertilized egg is imaged based on the set imaging height (step S104).

  Specifically, in the process of step S104, the imaging unit 103 acquires slice image data including information on the Z-axis height and the imaging time based on the imaging height. In particular, the imaging unit 103 acquires data indicating the Z-axis height from a predetermined position based on the imaging height indicating the height for simply adjusting the stage 101.

  For example, in the case of the first imaging, the imaging unit 103 images a fertilized egg at a height position (imaging height) that is 50 μm away from the reference position in the Z-axis direction, as shown in FIG. A slice image of name A1 and various data corresponding thereto are acquired.

  Next, the group ID assigning unit 104 assigns the same time group ID based on the imaging time in step S103 (step S105).

  Next, the communication control unit 106 transmits various data of the slice image generated by imaging together with the height at the time of imaging, the imaging time, and the group ID to the image processing apparatus 30 (step S106).

  When the image processing apparatus 30 receives various types of data including the slice image, the image processing apparatus 30 records the data in the image data recording unit 302 in association with the group ID.

  Next, the imaging unit 103 determines whether or not imaging has been completed (step S107). When it is determined that imaging has been completed (n = preset number of imaging times), imaging is completed and imaging is performed. If it is not determined that the processing has been completed (n <preset imaging count), the process returns to step S103.

In the example shown in FIG. 4, each piece of data of the slice image captured in this way is specifically captured five times while changing the height in 50 μm steps.
[6.2] Operation of Image Processing Device Next, the operation of the cell observation support process executed in the image processing device 30 of the present embodiment will be described with reference to FIG. FIG. 9 is a flowchart showing the operation of the cell observation support process executed in the image processing apparatus 30.

  This operation is executed after data of a plurality of slice images are recorded in the image data recording unit 302 of the image processing apparatus 30.

  Also, in this operation, a specific slice image is specified based on an operation specified by the user by operating the operation unit 370, and two slice images spatially adjacent to the specific slice image in the height direction. Is selected as the selected slice image.

  First, when the image designation unit 321 detects a user's operation for designating a specific slice image via the operation unit 370 (step S110), the image designation unit 321 designates a specific slice image based on the detected operation (step S111). For example, when the slice image shown in FIG. 4 is recorded in the image data recording unit 302 and the slice image A3 is designated as a specific slice image by the user, the image selection unit 322 displays the slice image A3. Is designated as the specific slice image A3.

  Next, the image specifying unit 321 reads data of all slice images having the same time group ID as the same time group ID of the specific slice image (step S112). For example, when the slice image shown in FIG. 4 is recorded in the image data recording unit 302, the image designating unit 321 reads the slice images A1 to A5.

  Next, the image selection unit 322 selects a slice image spatially adjacent to the specific slice image in the height direction as the selected slice image (step S113). For example, when the slice image shown in FIG. 4 is recorded in the image data recording unit 302 and the slice image A3 is designated as the specific slice image, the image selection unit 322 displays the specific slice image A3. In contrast, two slice images A2 and A4 spatially adjacent to each other in the height direction are selected as selected slice images.

  Next, the feature amount extraction unit 323 extracts feature amounts from the two selected slice images, and generates a feature amount image including information on the feature amounts (step S114). For example, when the slice images A2 and A4 illustrated in FIG. 4 are selected as the selected slice images, the image selection unit 322 generates the feature amount images of the selected slice images A2 and A4 illustrated in FIG. .

  Next, the image composition unit 324 generates a composite image by superimposing the feature amount image including the feature amount information extracted by the feature amount extraction unit 323 on the specific slice image (step S115). For example, when the slice image A3 shown in FIG. 4 is designated as the specific slice image and the slice images A2 and A4 are selected as the selected slice images, the image selection unit 322 is shown in FIG. A composite image 50 is generated.

  Next, the display control unit 350 performs predetermined image display control and causes the display unit 340 to display a composite image (step S116).

  Next, the image designation unit 321 determines whether the specific slice image has been changed based on the operation of the operation unit 370 during the display after the composite image is displayed or until the composite image is displayed after the specific slice image is designated. Is determined (step S117). At this time, if the image designation unit 321 determines that the specific slice image has been changed, the process returns to step S111. If it is determined that the specific slice image has not been changed, this operation ends.

  Note that the image selection unit 322 may select or change the selected slice image based on the operation of the operation unit 370.

[7] Modification [7.1] Modification 1
In the above embodiment, the first imaging time in the group is used as the same time group ID, but any time is used as long as it is within the range from the first imaging time to the last imaging time in the group. can do. Further, any ID may be used regardless of the imaging time.

[7.2] Modification 2
In the above-described embodiment, the height in the Z-axis direction may be expressed by, for example, absolute coordinates set on the stage 101, and a fixed position such as the bottom of the container that does not change temporally and spatially is used as a reference position. It may be expressed in relative coordinates.

[7.3] Modification 3
In the above-described embodiment, an example in which two slice images spatially adjacent to a specific slice image in the height direction are selected as selected slice images has been described, but the present invention is not limited to this. For example, one slice image that is spatially adjacent to the specific slice image in the height direction may be selected, or the imaging height may be two or three above, two below or three below, or the like. You may make it select the slice image imaged in (3).

  That is, at least one slice image captured at a time different from the specific slice image captured at a specific time may be used as the selected slice image.

  Alternatively, a plurality of candidates for the selected slice image may be displayed on the display unit 340, check boxes may be provided corresponding to each selected slice image, and a slice image for which the check box is checked may be used as the selected slice image. In this case, a configuration may be adopted in which a range of imaging heights and a time range are determined, and only slice images within the height and time ranges can be selected with check boxes.

[7.4] Modification 4
In the composite image 50 of the above-described embodiment, as shown in FIG. 6D, feature amount images 52 and 53 in slice images spatially adjacent to the specific slice image in the Z-axis direction are represented by the specific slice image. For example, the color of the low-side feature quantity image 52 is lightened and the color of the high-side feature quantity image 53 is darkened with respect to the Z-axis direction. The quantity image may be displayed as a line drawing, and the edge line of the feature quantity image may be displayed as a dotted line so that the vertical relationship in the height direction can be easily recognized by the color density or thickness of the dotted line. . In other words, any display of the feature amount image may be performed as long as the vertical relationship in the height direction can be easily recognized in the composite image.

  Further, the data processing unit 320 is provided with a resolution setting unit that sets the resolution of the image, the resolution setting unit sets the resolution of the feature amount image lower than the resolution of the specific slice image, and the display control unit 350 sets the feature amount. The image may be displayed on the display unit 340 so as to overlap the specific slice image. As a result, the feature amount image and the specific slice image can be clearly identified.

  Furthermore, when a fragment on the bottom surface side or ceiling surface side of an embryo is shown, it is preferable to indicate that it is a fragment by the color density or color tone attached to the fragment. For example, the position of the fragment can be easily confirmed by indicating that the fragment is closer to the bottom side as being reddish and closer to the ceiling side as being bluish.

[7.5] Modification 5
In the above embodiment, hyperlink information is embedded in the feature amount images 52 and 53 in the composite image 50 shown in FIG. 6D, and the user touches the feature amount image 53 on the screen of the display unit 340, for example. However, instead of the synthesized image 50, the selected slice image A4 shown in FIG. 6A may be displayed on the screen.

  In this case, the selected slice image A4 can be displayed as the specific slice image A4, and the selected slice images A3 and A5 spatially adjacent to the specific slice image A4 in the height direction can be displayed, or only the selected slice image A4 can be displayed. It can also be displayed.

[7.6] Modification 6
In the above embodiment, an example of grouping by the same time group ID is given, and the imaging height of each slice image is different from each other, but in observing the growth of the fertilized egg, the same height of the fertilized egg It is important to see how the state is changing.

  Therefore, instead of step S112 shown in FIG. 9, a process of reading slice image data of the same height having the same time group ID before and after the designated group from the image data recording unit 302 is performed.

  By this processing, for example, the data shown in FIG. 10 is obtained as the slice image data at a height of 150 μm.

  In FIG. 10, the same-time group ID of the slice image with the image name A3 is “001”, the same-time group ID of the slice image with the image name A8 is “002”, and the same-time group ID of the slice image with the image name A13 is “003”. The group imaging times are different from each other. In this example, if the slice image with the image name A8 is designated as the specific slice image A8, two slice images with the image name A3 and the image name A13 are obtained as two slice images adjacent to the specific slice image A8 in the time direction. The selected slice images A3 and A13 can be selected. As a result, a composite image obtained by combining the specific slice image A8 and the feature amount images of the selected slice images A3 and A13 can be acquired, and the growth state of the fertilized egg can be confirmed on one screen.

  In the description so far, spatial upper and lower connections and temporal connections have been described separately, but it is also possible to obtain a composite image that combines these two contents.

  For example, the upper and lower heights and the time before and after can be separated by coloring the feature amount image. For example, the top and bottom of the height are shown in green, and the time before and after is shown in red, or in addition to this, the height can be shown up and down or before and after the time.

  In this case, the lower side is green and lighter in color, the upper side is green and darker, and the time is red and lighter on the front side, and red and darker on the back side. In addition, the specific slice image indicating the current height and time can be displayed as a black and white image.

[7.7] Modification 7
In order to observe a fertilized egg, it is necessary to handle slice image data in units of fertilized eggs. For this purpose, a container ID can be assigned to each culture container for fertilized eggs and handled as one fertilized egg for each container, so that the container ID can be managed as a cell ID.

  Further, in a configuration in which the inside of the container is divided into a plurality of sections and an ID is assigned to each well in which the fertilized egg is cultured, a combination of the container ID and the well ID can be used as the fertilized egg ID. In addition, when culturing cells other than a fertilized egg, it will manage by cell ID.

  FIG. 11 shows an example when slice image data having the same fertilized egg ID is read from the image data recording unit 302. By displaying this data on the display unit 340, it is possible to allow the user to grasp that there are ten pieces of slice image data with the fertilized egg ID “0001-01” from image names A1 to A10. In addition, by displaying this data on the display unit 340 and allowing the user to specify any one specific slice image or select any one or two or more selected slice images, The feature amount image obtained from the selected slice image related thereto can be synthesized and displayed on one screen.

  Next, ID and fertilized egg ID will be described with reference to FIGS. 12 and 13.

  As shown in FIG. 12, an IC tag 71 is attached to the side surface of the fertilized egg culture container 70. If information on the container ID indicating the identification information of the culture container 70 is set in the IC tag 71, it can be distinguished from other culture containers.

  Further, as shown in FIG. 13, eight wells 81 to 88 for culturing fertilized eggs are provided in the culture container 70. The eight wells 81 to 88 can be distinguished from each other by a combination of dot marks formed in the vicinity of the wells. The dot mark is formed at any one of the four positions of the mark positions a to d. A dot mark is always formed at the mark position d, and the dot mark at this position is a reference for identification.

  In FIG. 13, for example, since the well 83 has dot marks formed at the mark positions b and d, the fertilized egg ID being cultured in the well 83 is “3”. Therefore, the fertilized egg being cultured in the well 83 can be distinguished from other fertilized eggs by the ID and fertilized egg ID in the container.

[7.8] Modification 8
In the above embodiment, the Z-axis direction perpendicular to the XY plane on which the container is arranged is described as the specific direction according to the present invention. However, the specific direction according to the present invention means only a direction completely perpendicular to the XY plane. Instead, the axial direction may intersect with the XY plane at an arbitrary angle.

  For example, the axis perpendicular to the XY plane (reference axis) is used as a reference in such a range that the contents such as fertilized eggs and culture fluid do not leak from the culture container and wells in the container and the slice image cannot be taken. In addition, an axial direction from +5 degrees to -5 degrees from the reference axis (that is, an axis in the range of 85 degrees to 95 degrees when the reference axis is 90 degrees with respect to the XY plane) can be applied as a specific direction. Even if the angle range is exceeded, if the specific slice image can be acquired, the axial direction can be used as the specific direction.

  As described above, in the cell observation support system 1 of the present embodiment, the display control unit 350 displays the feature amount image including the feature amount information of the selected slice image on the specific slice image on the display unit 340, so that the user desires The relationship between the specific slice image to be selected and the selected slice image can be recognized, and the state of the specific slice image and the selected slice image can be confirmed accurately.

  Therefore, the cell observation support system 1 according to the present embodiment can accurately confirm the state of the target cell while recognizing the relevance of the plurality of cell images in the target cell desired by the user.

DESCRIPTION OF SYMBOLS 1 Cell observation support system 10 Imaging device 20 Network 30 Image processing apparatus 40 Marker 300 Data recording part 301 Application recording part 302 Image data recording part 310 Communication control part 320 Data processing part 321 Image designation part 322 Image selection part 323 Feature-value extraction part 324 Image composition unit 340 Display unit 350 Display control unit 370 Operation unit 380 Management control unit

Claims (10)

In an image processing apparatus that processes a plurality of cell images in a target cell to be observed associated with at least one of a distance in a specific direction of a cell and a time at which the cell is imaged,
Specific cell image designating means for designating a specific specific cell image from the plurality of cell images;
Cell image selection means for selecting at least one cell image different from the specific cell image as a selected cell image from the plurality of cell images;
Feature quantity extraction means for extracting a feature quantity from the selected at least one selected cell image;
Display control means for displaying on the display means a feature amount image including information of the extracted feature amount on the specific cell image;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The plurality of cell images are associated with a distance in the specific direction;
The specific cell image designating unit designates a specific cell image captured at a specific distance,
The image processing apparatus, wherein the cell image selection unit selects at least one cell image captured at a distance different from the specific distance as the selected cell image. The image processing apparatus according to claim 1.
The plurality of cell images are associated with the time when the cells are imaged,
The specific cell image designating unit designates a specific cell image captured at a specific time,
The image processing apparatus, wherein the cell image selection unit selects at least one cell image captured at a time different from the specific time as the selected cell image. The image processing apparatus according to claim 1.
The plurality of cell images are associated with a distance in the specific direction and a time when the cells are imaged,
The specific cell image designating unit designates a specific cell image captured at a specific distance and a specific time,
The image processing apparatus, wherein the cell image selection unit selects, as the selected cell image, at least one cell image captured at a distance different from the specific distance or a time different from the specific time. The image processing apparatus according to claim 2 or 4,
The image processing apparatus, wherein the cell image selection unit selects at least one of two cell images spatially adjacent to the specific cell image in the specific direction. The image processing apparatus according to claim 3 or 4,
The image processing apparatus, wherein the cell image selection unit selects at least one of two cell images temporally adjacent to the specific cell image. The image processing apparatus according to any one of claims 1 to 6,
The cells are accommodated in a container in which a positioning portion that performs positioning in a surface direction of a surface orthogonal to the specific direction is formed,
Each of the plurality of cell images includes an image of at least a part of the positioning unit;
The display control means causes the display means to display the feature amount image on the specific cell image based on the image of the positioning unit. The image processing apparatus according to any one of claims 1 to 7,
The image processing apparatus, wherein the display control unit causes the display unit to display the feature amount image at a resolution lower than that of the specific cell image. In an image processing program for causing a computer to execute image processing for processing a plurality of cell images of an observation target associated with at least one of a distance in a specific direction of a cell and a time at which the cell is imaged,
The computer,
Specific cell image designating means for designating specific specific cell images from the plurality of cell images;
A cell image selection means for selecting at least one cell image different from the specific cell image as a selected cell image from the plurality of cell images;
A feature amount extraction means for extracting a feature amount from the selected at least one selected cell image; and
Display control means for displaying on the display means a feature amount image including information of the extracted feature amount on the specific cell image;
An image processing program that functions as an image processing program. In an image processing method for processing a plurality of cell images of an observation target associated with at least one of a distance in a specific direction of a cell and a time at which the cell is imaged,
Specify a specific cell image from the plurality of cell images,
Selecting at least one cell image different from the specific cell image from the plurality of cell images as a selected cell image;
Extracting features from the selected at least one selected cell image;
An image processing method comprising: displaying a feature amount image including information of the extracted feature amount on a display unit so as to overlap the specific cell image.

Images