JP2011192109A - Image processing apparatus, image processing method, program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately adjust the position of a fertilized ovum among a plurality of images when analyzing the temporal changes in the fertilized ovum. <P>SOLUTION: A reference profile creation program 2 defines a plurality of division areas along the accurate contours of the fertilized ovum in a reference image and calculates an amount of statistics of pixel values in the division areas to make a reference profile 6. A candidate profile creation program 3 calculates initial candidate contours on the basis of a reference contour, defines a plurality of division areas along the initial candidate contours, calculates other candidate contours by changing the positions and/or sizes of the initial candidate contours as well as calculates the amount of statistics of pixel values in the division areas, defines a plurality of division areas along the other candidate contours, and calculates an amount of statistics of pixel values in the division areas to prepare candidate profiles 7. A contour decision program 4 calculates similarity indicating the similarity between the reference profile 6 and the candidate profiles 7 to decide the contours of the fertilized ovum. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus or the like that analyzes an image obtained by photographing a fertilized egg during culture.

  Currently, fertilized eggs such as livestock are cultured in an incubator and grown until they are ready for transplantation. Since fertilized eggs with low quality are difficult to be fertilized even after transplantation, it is necessary to evaluate the quality of the fertilized eggs by some mechanism. In the case of in vitro fertilized eggs of cows, conventionally, an expert took out the fertilized egg from the incubator on the seventh day of culture and observed it with a microscope to evaluate the final quality of the fertilized egg. However, taking out and observing from the incubator is a great stress for a fertilized egg.

  Patent Document 1 discloses a fertilized egg quality evaluation support system that presents information useful for quality evaluation of a fertilized egg using only an image captured without taking out the fertilized egg from an incubator. In the technique described in Patent Document 1, a culture container in which a fertilized egg is arranged is periodically imaged, and a minimum approximate circular area including the fertilized egg is cut out from the captured image. And the information useful for quality evaluation of a fertilized egg is presented by analyzing the temporal change of the cut-out image.

Japanese Patent Application No. 2010-001013

In general, when an analysis target moves autonomously, in order to analyze a temporal change from an image, it is indispensable to align the analysis target among a plurality of images.
Conventional techniques for alignment include a method for comparing the shape of the entire contour to be analyzed, a method for comparing characteristic parts such as the transition of the pattern to be analyzed and the shape of part of the contour line, and a sample called a template. There is a method in which an image is compared with each image and alignment is performed based on a template having the highest degree of matching.
These conventional techniques exhibit sufficient performance for an image that can identify a boundary used as a reference for alignment processing, such as a contour of an analysis target or a pattern change. However, none of the conventional techniques can provide sufficient performance for fertilized eggs.

The fertilized egg is substantially spherical, and the surface portion of the sphere where the outline of the fertilized egg should appear is composed of a translucent zone that is a translucent, gelatinous shell having a certain thickness. In the zona pellucida, light repeatedly reflects and refracts in a complicated manner, and therefore, there are a part that becomes darker than the background, a part that becomes brighter than the background, and a part that has the same brightness as the background. In these parts, there is an unclear part that has the same brightness as the background due to the amount of light during imaging, and the boundary between the zona pellucida and the background (= the contour of the fertilized egg surface) is detected by a known image processing technique. It is difficult.
On the other hand, the fertilized egg contains countless portions called fat droplets that are difficult to transmit light (black dots in the image). The pixel value of the fat droplet is easily detected by a known image processing technique because there is a clear difference from the pixel values of other parts. However, the distribution of fat droplets is not uniform within the fertilized egg. Each fat droplet continues to move randomly. Therefore, the fat droplet in the fertilized egg image cannot be used as a reference for the alignment process. In addition, when trying to detect the contour of the fertilized egg surface by the above-described conventional technology, the inner contour of the fertilized egg derived from the fat droplet is too strong, which hinders processing.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to accurately match the position of a fertilized egg between a plurality of images when analyzing a time-dependent change of the fertilized egg. It is to provide a processing device and the like.

In order to achieve the above-described object, the first invention is an image processing apparatus for analyzing an image obtained by photographing a fertilized egg in culture, and images the same fertilized egg with different light source directions at different times. An arbitrary image from the input image group obtained as a reference image is used as a reference image, an accurate fertilized egg contour in the reference image is used as a reference contour, and a plurality of divided regions are defined along the reference contour. A reference profile creating means for calculating a statistic of pixel values in the region and storing the calculation result as a reference profile; an arbitrary image from the input image group as a processed image; and the processed image based on the reference contour Calculating a first candidate contour which is one of the contour candidates of the fertilized egg, defining a plurality of divided regions along the first candidate contour, and calculating a statistic of pixel values in the divided regions ,Calculation results A first candidate profile creating means for storing as a candidate profile, and a second candidate that is one of the contour candidates of the fertilized egg in the processed image by changing the position and / or size of the first candidate contour Second candidate profile creating means for calculating a contour, defining a plurality of divided regions along the second candidate contour, calculating a statistic of pixel values in the divided regions, and storing the calculation result as a candidate profile And a contour determining means for calculating a similarity indicating the similarity between the reference profile and the candidate profile, and determining a contour of a fertilized egg in the processed image based on a candidate profile most similar to the reference profile; An image processing apparatus comprising:
According to the first invention, the position of the fertilized egg can be accurately aligned between input images obtained by imaging the same fertilized egg at different times with the light source direction fixed.

A plurality of divided regions in the reference profile creating means, the first candidate profile creating means, and the second candidate profile creating means in the first invention are defined in a direction along the contour on the inside and outside of the contour, respectively. It is desirable that each of them has a certain width in a direction perpendicular to the contour.
Accordingly, the contour can be determined based on a change in pixel values around the contour with respect to the direction along the contour.

In addition, it is preferable that the similarity in the contour determination unit according to the first invention is calculated based on a difference in statistic values of pixel values in the inner divided region and the outer divided region facing each other through the contour.
Thus, the contour can be characterized using the contrast strength between the inside and outside of the contour.

Moreover, the said fixed width | variety in 1st invention is made into about half of the thickness of the zona pellucida which covers the fertilized egg estimated beforehand, for example.
Thus, since the inner divided region does not include the fertilized egg center, it is possible to eliminate the influence caused by the autonomous movement of the fat droplets at the fertilized egg center in the contour determining means.

Further, the fixed width in the first invention is, for example, a size of a fertilized egg based on the reference contour and a size of a fertilized egg central portion estimated by specifying a fat droplet in the reference image by threshold processing. Calculate based on
Thereby, an appropriate value according to the image can be set as a certain width.

In addition, it is desirable that the second candidate profile creating means in the first invention increases the number of candidate profile patterns as the difference between the captured time of the processed image and the culture start time increases.
Thus, the number of candidate profile patterns can be set to an appropriate number in accordance with an increase in the size change of the fertilized egg in the second half of the culture period.

Further, the first candidate profile creating means in the first invention, for example, assumes that the size of the first candidate contour is the same as the size of the reference contour, and moves the first candidate contour. The optimal position of the first candidate contour is calculated by performing a comparison process with the reference contour based on the internal pixel value.
As a result, the first candidate contour that is the initial value of the candidate for the contour of the fertilized egg can be automatically calculated.

Further, the first candidate profile creating means in the first invention, for example, assumes that the size of the first candidate contour is the same as the size of the reference contour, and moves the first candidate contour. The first candidate is compared with the reference contour based on pixel values of a region that includes the inside and outside of the first candidate contour and has a certain width in the direction perpendicular to the contour. The optimum position of the contour is calculated.
This makes it possible to automatically calculate the first candidate contour that is the initial value of the contour candidate of the fertilized egg without being affected by the autonomous movement of the fat droplets at the center of the fertilized egg. .

Further, the reference profile creating means in the first invention displays, for example, the reference image, and allows the user to designate the contour of the fertilized egg in the reference image as the reference contour.
Thereby, the reference contour in the reference image can be specified accurately.

Further, the reference profile creating means in the first invention designates a contour of a fertilized egg estimated by performing a template matching process between a pre-contained template image whose contour has been detected and the reference image as a reference contour.
As a result, the reference contour in the reference image can be automatically specified.

  Moreover, it is desirable that the profile comparison means in the first invention further generates an output image based on the determined outline of the fertilized egg. Thereby, an output image (fertilized egg observation image) according to the analysis content can be obtained.

The profile comparison means in the first invention calculates, for example, a rectangle that can enclose the contours of all times after determining the contours of input images at all times for the same fertilized egg, An output image is generated by matching the center of the rectangle with the center of the fertilized egg in the image and cutting out the input image along the rectangle.
Thereby, the image information of the site around the fertilized egg is not lost, and an output image that is easy to manage can be obtained.

2nd invention is an image processing method which makes the analysis object the image which image | photographed the fertilized egg in culture | cultivation, Comprising: The input image obtained by imaging the same fertilized egg with the light source direction fixed at different time An arbitrary image from the group is set as a reference image, an accurate fertilized egg outline in the reference image is set as a reference outline, a plurality of divided areas are defined along the reference outline, and a statistic of pixel values in the divided areas A reference profile creation step of calculating the calculation result and storing the calculation result as a reference profile; and an arbitrary image from the input image group as a processed image, and a candidate for a contour of a fertilized egg in the processed image based on the reference contour One first candidate contour is calculated, a plurality of divided regions are defined along the first candidate contour, a statistic of pixel values in the divided regions is calculated, and the calculation result is used as a candidate profile. A first candidate profile creation step that exists, and a position and / or size of the first candidate contour is changed to calculate a second candidate contour that is one of the contour candidates of the fertilized egg in the processed image. A second candidate profile creating step of defining a plurality of divided regions along the second candidate contour, calculating a statistic of pixel values in the divided regions, and storing the calculation result as a candidate profile; A contour determination step of calculating a similarity indicating a similarity between the profile and the candidate profile, and determining a contour of a fertilized egg in the processed image based on a candidate profile most similar to the reference profile. This is a featured image processing method.
According to the second invention, it is possible to accurately match the position of the fertilized egg between input images obtained by imaging the same fertilized egg at different times with the light source direction fixed.

A third invention is a program for causing a computer to function as the image processing apparatus of the first invention.
By distributing the third invention over a network and installing it on a general-purpose computer, the image processing apparatus of the first invention can be provided.

A fourth invention is a storage medium storing a program for causing a computer to function as the image processing apparatus of the first invention.
An image processing apparatus according to the first invention is provided by mounting the fourth invention on a storage medium reading device provided in a general-purpose computer and installing the program stored in the fourth invention in a general-purpose computer. Can do.

  According to the present invention, it is possible to provide an image processing apparatus or the like that can accurately match the position of a fertilized egg between a plurality of images when analyzing a time-dependent change of the fertilized egg.

Schematic diagram showing an image of a fertilized egg of 1-cell egg Schematic diagram showing an image of a two-cell egg fertilized egg The schematic diagram which shows the state on which the outline was superimposed on the image which imaged the fertilized egg of 1 cell egg The schematic diagram which shows the state on which the outline was superimposed on the image which imaged the fertilized egg of 2 cell egg Hardware configuration diagram of the image processing apparatus 1 Software configuration diagram of the image processing apparatus 1 Flow chart showing reference profile creation processing The figure explaining the 1st example of the process which designates the reference | standard outline 62 The figure explaining the 2nd example of the process which designates the reference | standard outline 62 The figure explaining the division area 63 Diagram explaining the process of determining a certain width The figure which shows the example of the reference | standard profile 6 Flow chart showing candidate profile creation processing The figure explaining the 1st example of the process which determines the initial value of the candidate outline 72 The figure explaining the 2nd example of the process which determines the initial value of the candidate outline 72 The figure which shows the 1st example of the candidate profile 7 The figure which shows the 2nd example of the candidate profile 7 The figure which shows the 3rd example of the candidate profile 7 Flow chart showing contour determination processing

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings referred to in the embodiments of the present invention, for easy understanding of the description, an image obtained by imaging a fertilized egg in culture that is an analysis target is schematically shown. However, it goes without saying that the present invention can be applied to an actual image.
In addition, the image schematically shown in the embodiment of the present invention is a grayscale image expressed only by light and darkness from white to black, but the present invention can be similarly applied to a so-called color image. Needless to say.

First, the characteristics of an image obtained by imaging a fertilized egg will be described with reference to FIGS.
FIG. 1 is a schematic diagram illustrating an image obtained by imaging a fertilized egg of a one-cell egg. As shown in FIG. 1, there is a portion (region having a pixel value close to black in FIG. 1) with a strong outline mainly derived from fat droplets at the center of the fertilized egg (fertilized egg center). To do. On the other hand, the fertilized egg is covered with a zona pellucida (a donut-shaped region having a gray pixel value in FIG. 1) having a certain thickness. In the zona pellucida, there is an unclear part where the outline of the fertilized egg has the same brightness as the background by adjusting the light during imaging.

Here, the definition of the term in embodiment of this invention is demonstrated.
The “contour of a fertilized egg” means the outer edge (outer circle of double circles) of a donut-shaped zona pellucida in an image (two-dimensional).
The “fertilized egg center” means a region inside the inner edge (inner circle of the double circle) of the donut-shaped zona pellucida in the image (two-dimensional).
“Size of fertilized egg” is a size that includes both the zona pellucida and the center of the fertilized egg. When the outer edge of the zona pellucida in the image (two-dimensional) is assumed to be a perfect circle, the radius, diameter, Or it shall mean an area, and when an ellipse is assumed, it shall mean a major axis and a minor axis, or an area.
“Size of fertilized egg center” means a radius, a diameter, or an area when the fertilized egg center in an image (two-dimensional) is assumed to be a perfect circle, and an ellipse. Means a major axis and a minor axis, or an area.

FIG. 2 is a schematic diagram showing an image obtained by imaging a fertilized egg of a two-cell egg. The image capturing time shown in FIG. 2 is later than the image capturing time shown in FIG. As time passes, the central part of the fertilized egg in the image of FIG. 2 is different from the image of FIG. 1, and the size of the fertilized egg also changes.
In particular, the region corresponding to the fat droplet is greatly different from the image of FIG. 1 and is unevenly distributed in the center of the fertilized egg. Therefore, the center of gravity of the region that can be calculated by the conventional technique does not coincide with the center point of the fertilized egg.
On the other hand, since the direction of the light source does not change for the zona pellucida, the same characteristics (brightness distribution, etc.) as the image in FIG. 1 are maintained.
In the embodiment of the present invention, images that do not change the direction of the light source at the time of imaging, such as the images in FIGS.

FIG. 3 is a schematic diagram showing a state in which an outline is superimposed on an image obtained by imaging a fertilized egg of a one-cell egg, and FIG. 4 is a schematic diagram showing a state in which an outline is superimposed on an image obtained by imaging a fertilized egg of a two-cell egg. FIG.
Both FIG. 3 and FIG. 4 show the outline of a fertilized egg expected to be detected from the images of FIG. 1 and FIG. In general, since a fertilized egg is a sphere, a circular outline is superimposed on both FIG. 3 and FIG. The center position of the circle shown in FIGS. 3 and 4 can be used as a common reference indicating the position of a fertilized egg between images taken at different imaging times. If the position of the fertilized egg can be accurately identified, it becomes possible to precisely analyze changes in the size of the fertilized egg, changes in the state of the fertilized egg, and the like.
In the embodiment of the present invention, the fertilized egg is assumed to be an ellipsoid, and an elliptical outline can be superimposed.

Next, an image processing apparatus according to an embodiment of the present invention will be described with reference to FIGS.
Below, in order to avoid confusion, a fertilized egg and a fertilized egg center part are demonstrated as a sphere.

First, the configuration of the image processing apparatus 1 will be described with reference to FIGS. 5 and 6.
FIG. 5 is a hardware configuration diagram of the image processing apparatus 1. Note that the hardware configuration in FIG. 5 is merely an example, and various configurations can be adopted depending on the application and purpose.

  In the image processing apparatus 1, a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, and the like are connected via a bus 18. The

  The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

The CPU calls and executes a program stored in the storage unit 12, ROM, recording medium, or the like to a work memory area on the RAM, drives and controls each device connected via the bus 18, and the image processing apparatus 1 The process to be described later is realized.
The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like.
The RAM is a volatile memory, and temporarily stores programs, data, and the like loaded from the storage unit 12, ROM, recording medium, and the like, and includes a work area used by the control unit 11 for performing various processes.

The storage unit 12 is an HDD (hard disk drive), and stores a program executed by the control unit 11, data necessary for program execution, an OS (operating system), and the like. With respect to the program, a control program corresponding to an OS (operating system) and an application program for causing a computer to execute processing described later are stored.
Each of these program codes is read by the control unit 11 as necessary, transferred to the RAM, read by the CPU, and executed as various means.

The media input / output unit 13 (drive device) inputs / outputs data, for example, media such as a CD drive (-ROM, -R, -RW, etc.), DVD drive (-ROM, -R, -RW, etc.) Has input / output devices.
The communication control unit 14 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between a computer and a network, and performs communication control between other computers via the network. The network may be wired or wireless.

The input unit 15 inputs data and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad.
An operation instruction, an operation instruction, data input, and the like can be performed on the computer via the input unit 15.
The display unit 16 includes a display device such as a CRT monitor and a liquid crystal panel, and a logic circuit (such as a video adapter) for realizing a video function of the computer in cooperation with the display device.

The peripheral device I / F (interface) unit 17 is a port for connecting a peripheral device to the computer, and the computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 17. Examples of the peripheral device include an imaging device for imaging a fertilized egg. The peripheral device I / F unit 17 is configured by USB, IEEE 1394, RS-232C, or the like, and usually includes a plurality of peripheral devices I / F. The connection form with the peripheral device may be wired or wireless.
The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

  FIG. 6 is a software configuration diagram of the image processing apparatus 1. The image processing apparatus 1 functions as various means by storing each program shown in FIG. 6 in the storage unit 12, reading each program code as necessary by the control unit 11, moving it to the RAM, and reading it to the CPU. .

  The image processing apparatus 1 stores the reference profile creation program 2, the candidate profile creation program 3, the contour determination program 4 and the like in the storage unit 12 as executable program codes. Further, the image processing apparatus 1 stores the input image 5, the reference profile 6, the candidate profile 7, the output image 8, and the like in the storage unit 12 or the RAM as the processing proceeds.

The input image 5 is an image group obtained by imaging the same fertilized egg with different light source directions at different times.
The reference profile creation program 2 is a program for realizing reference profile creation processing. In the reference profile creation process, an arbitrary image from the input images 5 is used as a reference image, an accurate fertilized egg contour in the reference image is used as a reference contour, and a plurality of divided regions are defined along the reference contour. The statistic of the pixel value is calculated, and the calculation result is stored as the reference profile 6. The reference profile creation process will be described later with reference to FIG.

  The candidate profile creation program 3 is a program for realizing candidate profile creation processing. In the candidate profile creation process, an arbitrary image from the input image 5 is used as a processed image, and an initial candidate contour (first candidate contour) that is one of the candidates for the contour of the fertilized egg in the processed image is determined based on the reference contour. Calculate, define a plurality of divided areas along the initial candidate contour, calculate the statistic of the pixel values in the divided areas, and store the calculation result as the candidate profile 7. Further, in the candidate profile creation processing, the position and / or size of the initial candidate contour is changed to calculate another candidate contour (second candidate contour) that is one of the contour candidates of the fertilized egg in the processed image. A plurality of divided regions are defined along other candidate contours, a statistic of pixel values in the divided regions is calculated, and the calculation result is stored as a candidate profile 7. The candidate profile creation process will be described later with reference to FIG.

  The contour determination program 4 is a program for realizing contour determination processing. In the contour determination process, the similarity indicating the similarity between the reference profile 6 and the candidate profile 7 is calculated, and the contour of the fertilized egg in the processed image is determined based on the candidate profile 7 that is most similar to the reference profile 6. Further, in the contour determination process, an output image 8 (fertilized egg observation image) is generated based on the determined contour of the fertilized egg. The contour determination process will be described later with reference to FIG.

Next, reference profile creation processing will be described with reference to FIGS.
FIG. 7 is a flowchart showing the reference profile creation process.
As shown in FIG. 7, the control unit 11 of the image processing apparatus 1 selects a reference image from the input image 5 (S101), and designates a reference contour (S102).

  FIG. 8 is a diagram for explaining a first example of processing for designating the reference contour 62. In the example illustrated in FIG. 8, the control unit 11 displays the reference image 61 on the display unit 16 and causes the user to select the contour of the fertilized egg in the reference image 61 as the reference contour 62. Specifically, the user enlarges, reduces, or moves a circle pointed by the mouse pointer 19 using a mouse that is one of the input units 15, the fertilized egg outline (size and position), and the mouse pointer. Match the size and position of the circle pointed to by 19. The control unit 11 calculates the coordinates and radius of the center point of the circle indicated by the mouse pointer 19 and stores them in the RAM or the like as the coordinates (xb, yb) and radius “rb” of the center point of the circle indicating the reference contour 62. The subscript “b” is an acronym for “base”.

FIG. 9 is a diagram for explaining a second example of processing for designating the reference contour 62. In the example shown in FIG. 9, the contour of the fertilized egg estimated by performing template matching processing between the template image 91 that has been detected in advance and the reference image 61 is designated as the reference contour 62.
The template image 91 may be stored in the storage unit 12 in advance, or may be input via the media input / output unit 13, the communication control unit 14, the peripheral device I / F unit 17 or the like as necessary. Also good. Moreover, the template image 91 used for the process may be determined in advance, or the control unit 11 may include a plurality of images based on predetermined conditions (conditions based on information such as the type of fertilized egg and culture environment). Or may be selected by the user.

As shown in FIG. 9, the template image 91 is superimposed with a template contour 92 which is an accurate contour. The circle indicating the template outline 92 has a center point (xt, yt) and a radius rt. The subscript “t” is an acronym for “template”.
A template matching process between the template image 91 and the reference image 61 may be performed using an appropriate method from known techniques. For example, the control unit 11 can compare all the pixels inside the template contour 92, some pixels inside the template contour 92, pixels around the template contour 92, and the like.

  When the reference contour 62 is designated by the process shown in FIG. 9, it is desirable to select an image captured in the initial stage of culture as the reference image 61. In the early stage of culture, the individual contour regarding the size of the fertilized egg and the state of the central part of the fertilized egg is small, and the reference contour 62 is accurately detected by selecting the template image 91 having the same type and culture environment of the fertilized egg. can do.

Returning to the description of FIG.
Next, the control unit 11 defines a plurality of divided regions inside and outside the reference contour 62 designated in S102 (S103). A plurality of divided regions are defined in the direction along the reference contour 62 on the inner side and the outer side of the reference contour 62, respectively, and have a certain width in the direction perpendicular to the reference contour 62.

FIG. 10 is a diagram for explaining the divided region 63.
As illustrated in FIG. 10, the control unit 11 defines a plurality of divided regions 63 having a certain width (indicated by 64 a in FIG. 10) in the direction perpendicular to the reference contour 62 inside the reference contour 62. In other words, first, a circle 65a (shown by a dotted line in FIG. 10) having a center coordinate equal to the reference contour 62 and a radius obtained by subtracting a constant value “w” from the radius “rb” of the reference contour 62 is defined. To do. Then, a donut-shaped double circular region having the reference contour 62 as an outer circle and the circle 65 a as an inner circle is divided into a plurality of divided regions 63.
Further, as illustrated in FIG. 10, the control unit 11 defines a plurality of divided regions 63 having a certain width (indicated by 64 b in FIG. 10) in the direction perpendicular to the reference contour 62 outside the reference contour 62. To do. In other words, first, a circle 65b (indicated by a dotted line in FIG. 10) having a center coordinate identical to the reference contour 62 and a radius obtained by adding a constant value “w” to the radius “rb” of the reference contour 62 is defined. To do. Then, a donut-shaped double circle region having the reference contour 62 as an inner circle and the circle 65 b as an outer circle is divided into a plurality of regions 63.
The constant value “w” may be the same value on the inner side and the outer side, or may be different values.

  In the example illustrated in FIG. 10, the control unit 11 defines eight dividing lines that pass through the center coordinates (xb, yb) of the reference contour 62 and divides the dividing line into eight equal parts on the inner side and the outer side. “Ri0” to “Ri7” shown in FIG. 10 are the inner divided regions 63a. Also, “Ro0” to “Ro7” shown in FIG. 10 are the inner divided regions 63b. The subscript “i” is an acronym for “in”, and “o” is “out”.

Needless to say, it can be generalized not only into eight equal parts but also into n equal parts (n is a natural number of 2 or more).
Further, not only the division is performed equally, but the division may be performed unevenly based on some standard. As can be seen with reference to FIGS. 1 and 2, the zona pellucida region continues to have the same level of brightness in a certain range along the circumferential direction, and has different brightness at a certain boundary. Focusing on this, the control unit 11 may scan the pixels in the circumferential direction so that a portion where a tone jump of a certain size occurs is used as the boundary of the divided region 63.

For example, the constant value “w” is preferably about half the thickness of the zona pellucida that covers the center of the fertilized egg. By making the thickness of the zona pellucida about half of the zona pellucida, that is, the inner divided region 63a does not include the fertilized egg center, the fat droplets in the fertilized egg center are autonomous in the contour determination process described later. The influence of moving to can be eliminated. For example, the thickness of the zona pellucida may be measured in advance for each type of fertilized egg and input by the user via the input unit 15.
Further, the constant value “w” (a constant width of the divided region 63) may be automatically determined by the control unit 11 based on the reference image 61.

FIG. 11 is a diagram illustrating a process for determining a certain width.
For the reference image 61, a reference contour 62 is determined in S102. “Rb” shown in FIG. 11 is the radius of the reference contour 62.
The control unit 11 specifies fat droplet pixels by threshold processing (binarization processing). Further, for example, the control unit 11 calculates a circle having the smallest radius among the circles including all the pixels of the fat droplets, and estimates and fertilizes the inside of the calculated circle (illustrated by vertical line hatching in FIG. 11). Let it be egg center part 67. “Rp” shown in FIG. 11 is the radius of the estimated fertilized egg center 67. Note that the subscript “p” is an acronym for “presumption”.
Then, the control unit 11 determines the radius “rp” of the estimated fertilized egg center 67 from the radius “rb” of the reference contour 62 (the size of the fertilized egg based on the reference contour 62). ) Is taken as “t”.
Since “t” can be estimated as the thickness of the zona pellucida (thickness), the control unit 11 determines “t / 2” as a constant value “w”.

Returning to the description of FIG.
Next, the control unit 11 calculates the statistic of the pixel value in the divided region 63 (S104), and stores the calculation result as the reference profile 6 in the storage unit 12 or the RAM (S105).
The statistic of the pixel value in the divided area 63 can be an average value, for example. In the example illustrated in FIG. 10, average values Ai0 to Ai7 and Ao0 to Ao7 are calculated for the pixel values in the divided regions Ri0 to Ri7 and Ro0 to Ro7, respectively, and a vector V = (Ai0,..., Ai7, Ao0,..., Ao7). The reference profile 6 includes the vector V, the center coordinates (xb, yb) of the reference contour 62, the radius “rb”, and the like.

In addition to the average value, a useful example will be described as the statistic of the pixel value in the divided area 63.
In the example shown in FIG. 10, average values Ai0 to Ai7 and Ao0 to Ao7 are calculated for the pixel values in the divided areas Ri0 to Ri7 and Ro0 to Ro7, respectively. Next, Aall = (Ai0 +... + Ai7 + Ao0 +... + Ao7) / 16 (average of average values) is calculated. Then, Bi0 = Ai0-Aall,..., Bi7 = Ai7-Aall, Bo0 = Ao0-Aall,..., Bo7 = Ao7-Aall are calculated, and the vector V = (Bi0,..., Bi7, Bo0. , ..., Bo7). As a result, even when the brightness of the entire image varies between the reference image 61 and the other input image 5 due to the variation of the imaging environment, the profile is appropriately determined in the contour determination process described in detail in FIG. Can be compared.

Further, the elements of the vector Vb may include a plurality of statistics as well as one statistics.
In the example shown in FIG. 10, average values Ai0 to Ai7, Ao0 to Ao7, and variances Vi0 to Vi7 and Vo0 to Vo7 are calculated for the pixel values in the divided regions Ri0 to Ri7 and Ro0 to Ro7, Vector V = (Ai0,..., Ai7, Ao0,..., Ao7, Vi0,..., Vi7, Vo0,..., Vo7). This makes it possible to compare profiles in more detail in the contour determination process described in detail in FIG.

  FIG. 12 is a diagram illustrating an example of the reference profile 6. FIG. 12 shows an image in which the reference contour 62 is superimposed on the image shown in FIG. 1 and the average value of the pixel values in each of the divided regions Ri0 to Ri7 and Ro0 to Ro7 so that it can be easily visually identified. A rectangle with corresponding brightness is shown.

Next, candidate profile creation processing will be described with reference to FIGS.
FIG. 13 is a flowchart showing candidate profile creation processing.
As shown in FIG. 13, the control unit 11 of the image processing apparatus 1 uses an arbitrary image (an image in which the outline of a fertilized egg is unknown) from the input image 5 as a processed image, and based on a reference contour 62, An initial candidate contour (first candidate contour) that is one of the contour candidates of the fertilized egg is calculated (S201).

  FIG. 14 is a diagram illustrating a first example of processing for determining the initial candidate contour 72. In the example shown in FIG. 14, the size of the initial candidate contour 72 is assumed to be the same as the size of the reference contour 62, and the initial candidate contour 72 is moved and compared with the reference contour 62 based on the pixel values inside the contour. By performing the processing, the optimum position of the initial candidate contour 72 is calculated.

For the reference image 61, the center coordinates (xb, yb) and the radius “rb” of the reference contour 62 are determined. The control unit 11 sets the inside of the reference contour 62 as the comparison area 68a. Further, the control unit 11 defines a temporary center coordinate (x0, y0) with the radius “r0” of the initial candidate contour 72 as the radius “rb” of the reference contour 62. The control unit 11 sets the inside of the initial candidate contour 72 as the comparison region 78a.
The control unit 11 calculates the sum of square errors for each pixel in the comparison region 68a and the comparison region 78a while shifting the center coordinates (x0, y0) of the initial candidate contour 72, and the sum of the square errors is calculated. The minimum center coordinate (x0, y0) is determined.

  FIG. 15 is a diagram illustrating a second example of processing for determining the initial candidate contour 72. In the example shown in FIG. 15, it is assumed that the size of the initial candidate contour 72 is the same as the size of the reference contour 62, and the initial candidate contour 72 is moved in the direction perpendicular to the contour, including the inside and outside of the contour. The optimum position of the initial candidate contour 72 is calculated by performing a comparison process with the reference contour 62 based on the pixel values of the regions each having a certain width.

In the reference image 61, the center coordinates (xb, yb) and the radius “rb” of the reference contour 62 have been determined. The control unit 11 sets an area including the inside and outside of the reference outline 62 and having a certain width in the direction perpendicular to the outline as a comparison area 68b. Further, the control unit 11 defines a temporary center coordinate (x0, y0) with the radius “r0” of the initial candidate contour 72 as the radius “rb” of the reference contour 62. The control unit 11 sets a region including the inside and the outside of the initial candidate contour 72 and having a certain width in the direction perpendicular to the contour as a comparison region 78b.
Then, the control unit 11 calculates the sum of square errors for each pixel in the comparison region 68b and the comparison region 78b while shifting the center coordinates (x0, y0) of the initial candidate contour 72, and the sum of the square errors is calculated. The minimum center coordinate (x0, y0) is determined.
The constant width is set such that the comparison regions 68b and 78b do not include the central region of the fertilized egg (for example, the constant value “w” described above in S103 of FIG. 7). Thereby, it is possible to eliminate the influence caused by the autonomous movement of the fat droplets at the center of the fertilized egg.

Returning to the description of FIG.
Next, the control unit 11 defines a plurality of divided regions inside and outside the initial candidate contour 72 (S202). A plurality of divided regions are defined in the direction along the initial candidate contour 72 on the inner side and the outer side of the initial candidate contour 72, respectively, and have a certain width in the direction perpendicular to the initial candidate contour 72.
The definition of the divided area in the processed image (S202) is performed in the same manner as the definition of the divided area in the reference image 61 (S103 in FIG. 7).

  Next, the control unit 11 calculates the statistic of the pixel value in the divided area (S203), and stores the calculation result as the candidate profile 7 in the storage unit 12 or the RAM (S204). The calculation of the statistic of the pixel value in the divided area in the processed image (S203) is performed in the same manner as the calculation of the statistic of the pixel value in the divided area in the reference image 61 (S104 in FIG. 7).

Next, the control unit 11 checks whether all patterns of the candidate profile have been calculated (S205). The candidate profile pattern will be described later in S206.
If all the patterns have not been calculated (No in S205), the control unit 11 proceeds to S206.
If all patterns have been calculated (Yes in S205), the control unit 11 ends the process.

In S206, the control unit 11 changes the candidate contour. The change of the candidate contour is to calculate a candidate contour different from the initial candidate contour 72 by changing the position and / or size of the initial candidate contour 72 calculated in 201.
Specifically, the control unit 11 changes, for example, ± 1 for three of the x coordinate “x0”, the y coordinate “y0”, and the radius “r0” of the center coordinates of the initial candidate contour 72, and all combinations Are calculated as candidate profile patterns. When changing by ± 1, 3 × 3 × 3 = 27 candidate profile patterns are calculated. “27 ways” is the number of patterns including the initial candidate area 72.

16 to 18 are diagrams illustrating a first example, a second example, and a third example of the candidate profile 7, respectively.
When the candidate contours superimposed on the images shown in FIGS. 16 and 17 are compared, the center coordinates and the radius are different. Further, when the candidate contours superimposed on the images shown in FIGS. 16 and 18 are compared, at least the center coordinates are different.
Since the central coordinates and radius are different, it can be seen that the average values of the pixel values in the divided regions Ri0 to Ri7 and Ro0 to Ro shown in FIGS. 16 to 18 also show different brightness.

In the above-described example, the x coordinate “x0”, the y coordinate “y0”, and the radius “r0” of the center coordinates of the initial candidate region 72 are changed by ± 1, but the change width is widened, For example, it may be ± 2. However, if the range to be changed is too wide, the pattern of candidate profiles becomes enormous and the processing time becomes slow.
Therefore, it is desirable to increase the number of candidate profile patterns based on appropriate criteria. For example, since the change in the size of the fertilized egg becomes more intense in the second half of the culture period, it is desirable to increase the number of candidate profile patterns as the difference between the processing image capturing time and the culture start time increases.

Returning to the description of FIG.
After S206, the process is repeated from S202. The control unit 11 defines a plurality of divided regions inside and outside the changed candidate contour (S202). A plurality of divided regions are defined in the direction along the changed candidate contour on the inner side and the outer side of the changed candidate contour, and have a certain width in the direction perpendicular to the changed candidate contour. Next, the control unit 11 calculates the statistic of the pixel value in the divided area (S203), and stores the calculation result as the candidate profile 7 in the storage unit 12 or the RAM (S204).
Thereafter, S206 and S202 to S204 are repeated until it is determined in S205 that all patterns of the candidate profile have been calculated.

Next, the contour determination process will be described with reference to FIG.
FIG. 19 is a flowchart showing the contour determination process.
As shown in FIG. 19, the control unit 11 of the image processing apparatus 1 selects one candidate profile 7 (S301), and calculates the similarity between the reference profile 6 and the candidate profile 7 (S302).

The degree of similarity is defined based on, for example, the sum Σ1 of square errors between the elements of the vectors (statistics for each divided region 63) included in both the reference profile 6 and the candidate profile 7. The pair for which the square error is obtained is divided areas 63 at the same position.
In this case, for example, the vector of the reference profile 6 is Vb = (Bi0,... Bin, Bo0,... Bon), and the vector of the candidate profile 7 is Vc = (Ci0,..., Cin, Co0,. .., Con) (n is a natural number of 2 or more) [Sigma] 1 = (Bi0-Ci0) ^ 2 + ... + (Bin-Cin) ^ 2 + (Bo0-Co0) ^ 2 + ... + (Bon-Con ) ^ 2.

In addition, another useful similarity is defined based on the sum Σ2 of square errors related to the difference between the statistic of the inner divided region 63a and the statistic of the outer divided region 63b facing each other through the contour. .
In this case, for example, the vector of the reference profile 6 is Vb = (Bi0,... Bin, Bo0,... Bon), and the vector of the candidate profile 7 is Vc = (Ci0,..., Cin, Co0,. .., Con) (n is a natural number of 2 or more) [Sigma] 2 = (| Bi0-Bo0 |-| Ci0-Co0 |)) ^ 2 + ... + (| Bin-Bon |-| Cin-Con |) ) ^ 2 (absolute symbol may be removed).
Using Σ2 as the similarity means that the contour is characterized using the contrast strength between the inside and outside of the contour.

Next, the control unit 11 confirms whether all candidate profiles 7 have been processed (S303).
When not all candidate profiles 7 have been processed (No in S303), the control unit 11 repeats the process from S301.
When all candidate profiles 7 have been processed (Yes in S303), the control unit 11 proceeds to S304.

  Next, the control unit 11 determines, based on the calculation result of S302, the candidate profile 7 that is most similar to the reference profile 6, that is, the candidate profile 7 that has the highest similarity (the value of the above-described Σ1 or Σ2 is the smallest). The contour of the processed image selected in S201 of FIG. 13 is determined (S304). The processed image is an image in which the outline of the fertilized egg selected from the input image 5 is unknown.

And the control part 11 produces | generates the output image 8 based on the determined outline (S305).
For example, after determining the contours for the input images 5 of all photographing times for the same fertilized egg, the control unit 11 determines the contour of which photographing time (that is, the outer edge of the donut-shaped zona pellucida covering the fertilized egg). Also, a rectangle having a size common to all shooting times that can be surrounded with some margin is calculated. And the control part 11 produces | generates the output image 8 by making the center of a rectangle and the center of a fertilized egg correspond in the input image 5 of each imaging | photography time, and cutting out the input image 5 along a rectangle, and memorize | stores in the memory | storage part 12. Remember. Further, the control unit 11 stores the determined contour information (information such as radius and center coordinates) in the storage unit 12 in combination with the output image 8. As a result, it is possible to obtain an output image 8 (fertilized egg observation image) that is easy to manage without losing the image information of the region around the fertilized egg.

Further, for example, the control unit 11 may generate the output image 8 by superimposing the determined contour on the processed image.
For example, the control unit 11 may generate the output image 8 by cutting out the inside of the outline of the fertilized egg determined from the processed image.
In addition, the control part 11 can produce | generate the output image 8 by various methods based on the determined outline according to the analysis content.

The image processing apparatus 1 uses all images (the input image 5 except the reference image 6) whose contours of the fertilized eggs are unknown as processed images, and performs the above processing to change the light source direction of the same fertilized eggs at different times. It is possible to generate an output image 8 in which the position of the fertilized egg is accurately matched with respect to all the images of the input image 5 obtained by imaging with a constant value.
The reference image 6 does not need to be fixed, and a new image may be designated as the reference image 6 as time elapses from the culture start time. In this case, the output image 8 having the contour determined by the process shown in FIG.

In the description of FIGS. 5 to 19 described above, the fertilized egg and the central portion of the fertilized egg are assumed to be a sphere, but the present invention can also be applied assuming an ellipsoid.
When ignoring the rotation of the fertilized egg, the above-described processing can be performed in substantially the same manner as the processing for the circle by replacing the radius parameter in the circle with the two parameters of the major axis and the minor axis of the ellipse. it can.
When the fertilized egg is assumed to be an ellipsoid, there are four parameters for generating the pattern of the candidate profile 8: the x coordinate of the center coordinate, the y coordinate, the major axis, and the minor axis.

Further, the technical idea of the present invention can be extended when considering the rotation of a fertilized egg. When the fertilized egg is rotated by the rotation angle θ, the angle measurement start point when defining the divided region 63 (in the example of FIG. 10, the circumferential ends of the divided regions “Ri0” and “Ro0”) Part) according to the rotation angle θ.
When the fertilized egg is assumed to be an ellipsoid and the rotation of the fertilized egg is taken into consideration, the parameters for generating the pattern of the candidate profile 8 are the five coordinates of the center coordinate x coordinate, y coordinate, major axis, minor axis, and rotation angle. .

When there is a change in the statistics around the contour due to the rotation, it is necessary to model the change in the statistics due to the rotation. That is, it is necessary to apply the rotation angle θ to the reference profile 6 and compare with the candidate profile 8.
As an example of the modeling method, rotation angles θ = 0, x, 2x, 3x,... (X is a divisor of 360, for example) are provided at predetermined intervals, and an elliptical shape is set for each rotation angle. A method of calculating the central coordinates, the major axis, the minor axis, and the statistics of the divided region 63, generating a plurality of reference profiles 6, and linearly interpolating them.
Thereby, it is possible to suppress the influence of the change in the statistics around the contour caused by the rotation.

  Furthermore, the center of gravity of the region inside the contour is used instead of the center coordinate of the circle or ellipse, and the area of the region inside the contour is used instead of the radius, major axis, or minor axis of the circle or ellipse. It is also possible to extend the technical idea of the invention. The case where the rotation is taken into consideration is the same as the case of the above ellipse.

  The preferred embodiments of the image processing apparatus and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

DESCRIPTION OF SYMBOLS 1 ......... Image processing apparatus 2 ......... Reference profile creation program 3 ......... Candidate profile creation program 4 ......... Outline determination program 5 ......... Input image 6 ......... Reference profile 7 ......... Candidate profile 8 ... …… Output image 61 ……… Reference image 62 ………… Reference contour 63 ………… Division area 71 ………… Processed image 72 ………… Initial candidate contour 91 ………… Template image 92 ……… Template contour

Claims (15)

An image processing apparatus for analyzing an image obtained by photographing a fertilized egg in culture,
Arbitrary image from the input image group obtained by imaging the same fertilized egg with different light source directions at different times as a reference image, the exact contour of the fertilized egg in the reference image as a reference contour, Defining a plurality of divided areas along the reference contour, calculating a statistic of pixel values in the divided areas, and storing a calculation result as a reference profile;
An arbitrary image from the input image group is used as a processed image, a first candidate contour that is one of the contour candidates of a fertilized egg in the processed image is calculated based on the reference contour, and the first candidate Defining a plurality of divided areas along the contour, calculating a statistic of pixel values in the divided areas, and storing a calculation result as a candidate profile;
A position and / or size of the first candidate contour is changed to calculate a second candidate contour that is one of the contour candidates of the fertilized egg in the processed image, and along the second candidate contour A second candidate profile creating means for defining a plurality of divided areas, calculating a statistic of pixel values in the divided areas, and storing the calculation result as a candidate profile;
A contour determining means for calculating a similarity indicating similarity between the reference profile and the candidate profile, and determining a contour of a fertilized egg in the processed image based on a candidate profile most similar to the reference profile;
An image processing apparatus comprising:   A plurality of divided regions in the reference profile creating means, the first candidate profile creating means, and the second candidate profile creating means are defined in a direction along the contour on the inner side and the outer side of the contour, and are perpendicular to the contour. The image processing apparatus according to claim 1, wherein each of the image processing apparatuses has a certain width in each direction.   The degree of similarity in the contour determination unit is calculated based on a difference in statistic values of pixel values in an inner divided region and an outer divided region facing each other through the contour. Image processing device.   The image processing apparatus according to claim 2, wherein the certain width is approximately half the thickness of a zona pellucida that covers a fertilized egg estimated in advance.   The fixed width is calculated based on a size of a fertilized egg based on the reference contour and a size of a fertilized egg central portion estimated by specifying a fat droplet in the reference image by threshold processing. The image processing apparatus according to claim 4.   The image processing apparatus according to claim 1, wherein the second candidate profile creating unit increases the number of candidate profile patterns as a difference between an imaging time of the processed image and a culture start time increases.   The first candidate profile creation means assumes that the size of the first candidate contour is the same as the size of the reference contour, and moves the first candidate contour based on the internal pixel value. The image processing apparatus according to claim 1, wherein an optimal position of the first candidate contour is calculated by performing a comparison process with a reference contour.   The first candidate profile creating means assumes that the size of the first candidate contour is the same as the size of the reference contour, and moves the first candidate contour while moving the first candidate contour. And calculating the optimum position of the first candidate contour by performing a comparison process with the reference contour based on pixel values of a region including a certain width in a direction perpendicular to the contour. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 1, wherein the reference profile creation unit displays the reference image and causes a user to designate a contour of a fertilized egg in the reference image as a reference contour.   The reference profile creating means designates a contour of a fertilized egg estimated by performing template matching processing between a pre-contained contour-detected template image and the reference image as a reference contour. An image processing apparatus according to 1.   The image processing apparatus according to claim 1, wherein the profile comparison unit further generates an output image based on the determined outline of a fertilized egg.   The profile comparison means calculates a rectangle that can enclose the contours of all times after determining the contours of the input images of all times for the same fertilized egg, and the center of the rectangle in each input image The image processing apparatus according to claim 11, wherein an output image is generated by matching a center of a fertilized egg and cutting out an input image along the rectangle. An image processing method for analyzing an image obtained by photographing a fertilized egg in culture,
Arbitrary image from the input image group obtained by imaging the same fertilized egg with different light source directions at different times as a reference image, the exact contour of the fertilized egg in the reference image as a reference contour, Defining a plurality of divided areas along the reference contour, calculating a statistic of pixel values in the divided areas, and storing a calculation result as a reference profile;
An arbitrary image from the input image group is used as a processed image, a first candidate contour that is one of the contour candidates of a fertilized egg in the processed image is calculated based on the reference contour, and the first candidate Defining a plurality of divided areas along the contour, calculating a statistic of pixel values in the divided areas, and storing a calculation result as a candidate profile;
A position and / or size of the first candidate contour is changed to calculate a second candidate contour that is one of the contour candidates of the fertilized egg in the processed image, and along the second candidate contour A second candidate profile creating step of defining a plurality of divided areas, calculating a statistic of pixel values in the divided areas, and storing the calculation result as a candidate profile;
A contour determination step of calculating a similarity indicating similarity between the reference profile and the candidate profile, and determining a contour of a fertilized egg in the processed image based on a candidate profile most similar to the reference profile;
An image processing method comprising:   A program for causing a computer to function as the image processing apparatus according to claim 1.   A storage medium storing a program for causing a computer to function as the image processing apparatus according to claim 1.