WO2020149376A1

WO2020149376A1 - Image processing device, image processing method, and program

Info

Publication number: WO2020149376A1
Application number: PCT/JP2020/001354
Authority: WO
Inventors: 吉田　隆彦; 萌伽信田
Original assignee: 株式会社ニコン
Priority date: 2019-01-16
Filing date: 2020-01-16
Publication date: 2020-07-23
Also published as: JP2020113200A; JP7383880B2

Abstract

An image processing device that connects fragmented cell regions in an image to extract a cell, said image processing device equipped with: an acquisition unit for acquiring a fragmented cell region from within the image; a first pixel setting unit for setting a first pixel in the fragmented cell region; and an extraction unit for setting a plurality of pixel groups on the basis of the position of the first pixel, selecting a second pixel on the basis of the luminance values of the pixels included in the plurality of pixel groups, and then connecting the first pixel and the second pixel, thereby extracting a cell.

Description

Image processing apparatus, image processing method and program

The present invention relates to an image processing device, an image processing method, and a program.

A method has been proposed in which image processing is performed on an image obtained by picking up an image of an object including a linear portion that appropriately includes branches and the like (see Patent Document 1). In such image processing, it is desirable to perform the processing promptly while suppressing deterioration in accuracy.

U.S. Patent No. 9646194

According to the first aspect of the present invention, an image processing apparatus is an image processing apparatus that connects fragmented cell regions in an image to extract cells, and obtains the fragmented cell region from the image. An acquisition unit, a first pixel setting unit that sets a first pixel in the fragmented cell region, and a plurality of pixel groups that are set based on the position of the first pixel and are included in the plurality of pixel groups. A second pixel is selected based on a brightness value of the pixel to be extracted, and then an extraction unit that connects the first pixel and the second pixel to extract a cell.
According to the second aspect of the present invention, an image processing method is an image processing method for extracting cells by connecting fragmented cell regions in an image, and obtaining fragmented cell regions from the image. Area acquisition, a first pixel setting that sets a first pixel in the fragmented cell area, and a plurality of pixel groups are set based on the position of the first pixel, and the plurality of pixel groups are included in the plurality of pixel groups. Selecting a second pixel based on the brightness value of the pixel, and then connecting the first pixel and the second pixel to extract cells to extract cells.
According to a third aspect of the present invention, a program is a program for connecting cells fragmented in an image to extract cells, and an acquisition process for acquiring a fragmented cell region from the image. A first pixel setting process for setting a first pixel in the fragmented cell region, and a plurality of pixel groups set based on the position of the first pixel, and brightness of pixels included in the plurality of pixel groups The processing device is caused to select the second pixel based on the value, and then perform the extraction process of connecting the first pixel and the second pixel to extract the cell.

FIG. 1 is a conceptual diagram showing a configuration of an image generating apparatus according to an embodiment. FIG. 2 is a conceptual diagram showing the configuration of the data processing unit. FIG. 3 is a conceptual diagram showing a probability distribution image. FIG. 4 is a conceptual diagram showing a connected area image. FIG. 5 is a conceptual diagram for explaining the search area. FIG. 6 is a conceptual diagram for explaining the setting of the starting point pixel. FIG. 7 is a conceptual diagram for explaining the selected pixel setting process. FIG. 8 is a conceptual diagram for explaining candidate pixels. FIG. 9 is a graph showing the relationship between luminance and candidate pixel parameters. FIG. 10 is a conceptual diagram showing a connected pixel area. FIG. 11 is a flowchart showing the flow of the image generation method according to the embodiment. FIG. 12 is a flowchart showing the flow of the image generation method according to the embodiment. FIG. 13 is a conceptual diagram for explaining re-extraction of selected pixels. FIG. 14 is a flowchart showing the flow of the image generation method according to the modification. FIG. 15 is a conceptual diagram for explaining the relationship between the size of the calculated pixel area and the width of the neurite. FIG. 16 is a conceptual diagram showing the configuration of the image generating apparatus according to the embodiment. 17(A), 17(B), 17(C) and 17(D) are conceptual diagrams for explaining the candidate area and the finalized area. FIG. 18 is a flowchart showing the flow of the image generation method according to the embodiment. FIG. 19 is a conceptual diagram for explaining the setting of the starting point pixel. FIG. 20 is a conceptual diagram showing the configuration of the image generation apparatus of the modified example. FIG. 21 is a conceptual diagram showing the configuration of the image generation apparatus of the modified example. 22(A), 22(B) and 22(C) are conceptual diagrams for explaining the detection of a crossover. 23(A), 23(B), 23(C) and 23(D) are conceptual diagrams for explaining the detection of a crossover. FIG. 24 is a conceptual diagram showing the configuration of the image generation apparatus of the modified example. FIG. 25 is a conceptual diagram for explaining the calculation of the width of the neurite. FIG. 26 is a conceptual diagram for explaining the calculation of the width of the neurite. FIG. 27 is a conceptual diagram for explaining the calculation of the output brightness value. FIG. 28 is a conceptual diagram showing the configuration of the image generating apparatus according to the embodiment. FIG. 29 is a conceptual diagram for explaining the first selected pixel setting process. FIG. 30 is a conceptual diagram for explaining the second selected pixel setting process. FIG. 31 is a flowchart showing the flow of the image generation method according to the embodiment. FIG. 32 is a flowchart showing the flow of the image generation method according to the embodiment. FIG. 33 is a conceptual diagram showing the starting point arrangement area. FIG. 34 is a flowchart showing the flow of the image generation method according to the modification. FIG. 35 is a conceptual diagram for explaining the provision of the program.

Hereinafter, modes for carrying out the present invention will be described with reference to the drawings.

-First embodiment-
The image generation apparatus of the first embodiment extracts a plurality of pixels based on the acquired image data (hereinafter, referred to as input image data), and the image data (hereinafter referred to as an output image) based on the extracted plurality of pixels. Data).

FIG. 1 is a conceptual diagram showing the configuration of the image generating apparatus of this embodiment. The image generation device 1 includes a culture unit 100 and an information processing unit 40. The culture unit 100 includes an incubator 10, an observation sample stage 11, a drive unit 12, and an imaging unit 20. The information processing unit 40 includes an input unit 41, a communication unit 42, a storage unit 43, an output unit 44, and a control unit 50. The control unit 50 includes a data processing unit 51, an output control unit 52, and a device control unit 53.

The image generation device 1 is configured as a culture device, and has a configuration in which input image data obtained by imaging in the culture unit 100 is input to the data processing unit 51 and processed.
Note that the image generating device 1 does not have to have a configuration in which the image generating device 1 performs imaging and culturing as long as it can acquire input image data.

In this embodiment, a linear portion including a straight line, a curved line, an intersection, or the like is extracted as an extraction target from the input image corresponding to the input image data. In the following, an explanation will be given using an example in which the cell Ce is a nerve cell and a portion corresponding to the neurite Nr is extracted, but the extraction target is not limited to this example as long as the linear portion as described above is included. , Can be any element that constitutes an image.

The culturing unit 100 cultivates the cell Ce and images the cultivated cell Ce.

The incubator 10 stores therein a culture container C in which cells Ce are cultured. The inside of the incubator 10 is controlled by a temperature controller (not shown) so that the culture is performed in a preset environment, such as being maintained at a preset temperature. The drive unit 12 includes an actuator, moves the culture container C at a predetermined time, and mounts it on the observation sample table 11 inside the incubator 10. Further, the driving unit 12 moves the imaging unit 20 or the observation sample stage 11 or the like to an appropriate position so that the cells Ce are arranged on the focal plane of the imaging unit 20 for imaging the cells Ce.

The image pickup unit 20 includes an image pickup device including an image pickup device such as a CMOS or CCD, and picks up an image of the cell Ce, particularly the neurite Nr of the cell Ce. The method of imaging by the imaging unit 20 is not particularly limited as long as the pixel corresponding to the neurite Nr in the captured image can be distinguished from other portions with desired accuracy by the brightness value of the pixel. For example, as a method of imaging by the imaging unit 20, a fluorescence observation method, a phase difference observation method, or the like can be used.

When the imaging unit 20 performs imaging by a fluorescence observation method, gene transfer causes a cell Ce to express a fluorescent protein such as GFP or a protein in which a protein localized in neurite Nr and a fluorescent protein are fused. By doing so, fluorescent staining can be performed. If there is no problem in using the cells Ce after imaging, another labeling method such as immunostaining may be performed.

Data corresponding to the captured image obtained by the image capturing unit 20 capturing the cell Ce is converted into a digital signal and input to the information processing unit 40 as input image data in which pixels are associated with brightness values (arrows). A1) is stored in the storage unit 43.

The information processing unit 40 serves as an interface with a user of the image generating apparatus 1 (hereinafter, simply referred to as “user”), and also performs processing such as communication, storage, and calculation regarding various data.
The information processing unit 40 may be configured as an information processing device that is physically separated from the culture unit 100. Further, a part of the data used by the image generating apparatus 1 may be stored in a remote server or the like, and a part of the arithmetic processing performed by the image generating apparatus 1 may be performed by a remote server or the like.

The input unit 41 includes an input device such as a mouse, a keyboard, various buttons or a touch panel. The input unit 41 receives, from a user, data necessary for imaging by the culture unit 100 and data processing by the data processing unit 51.

The communication unit 42 includes a communication device capable of communicating by wireless or wired connection such as the Internet, and appropriately transmits/receives data regarding control and processing in the image generation device 1.

The storage unit 43 includes a non-volatile storage medium, and stores a program that causes the control unit 50 to perform processing, image data regarding the processing of the data processing unit 51, and the like.

The output unit 44 includes a display device such as a liquid crystal monitor, and outputs an image or the like based on the output image data obtained by the processing of the data processing unit 51.

The control unit 50 is configured by a processing device such as a CPU, functions as a main body of the operation that controls the image generation device 1, and executes various processes by executing a program installed in the storage unit 43.

The data processing unit 51 of the control unit 50 processes the input image data input from the imaging unit 20, extracts a plurality of pixels corresponding to an extraction target, and outputs output image data (described later) based on the extracted pixels. Connected region image data) is generated.

FIG. 2 is a conceptual diagram showing the configuration of the data processing unit 51. The data processing unit 51 includes a probability distribution image generation unit 511, a region setting unit 512, an end point pixel setting unit 513, a start point pixel setting unit 514, a pixel extraction unit 600, and an image generation unit 700. The pixel extraction unit 600 includes a candidate pixel designation unit 601 and a selected pixel setting unit 602.

The probability distribution image generation unit 511 of the data processing unit 51 generates and acquires probability distribution image data corresponding to the probability distribution image based on the input image data stored in the storage unit 43.

FIG. 3 is a conceptual diagram showing the probability distribution image Gp. The probability distribution image Gp is an image in which the brightness of each pixel of the probability distribution image Gp is associated with the probability that the pixel corresponds to the neurite Nr to be extracted. FIG. 3 shows the probability distribution image Gp corresponding to a nerve cell, but for the sake of clarity, only one neurite Nr is shown. In the probability distribution image Gp of FIG. 3, the darker the hatched portion, the higher the possibility that the pixel corresponding to that portion is the image portion corresponding to the neurite Nr. In the probability distribution image Gp, it is preferable that the above probability corresponds to each pixel as a one-dimensional brightness value and is expressed as a gray scale image, but if the above probability and brightness value are associated with each other, the probability distribution image Gp The aspect of is not particularly limited.

The probability distribution image generation unit 511 calculates the probability corresponding to the neurite Nr, which is the extraction target, for each pixel by processing the input image data with a predetermined image processing algorithm. The predetermined image processing algorithm is learned machine learning. This machine learning is deep learning in which a plurality of images obtained by imaging nerve cells and an image showing a portion corresponding to a neurite in the image are input to and learned by an arithmetic device.
The predetermined image processing algorithm is not particularly limited as long as the probability and the brightness value are associated with each other, and machine learning other than deep learning or an algorithm other than machine learning may be used.

In the probability distribution image Gp, unlike the cell Ce of the subject (FIG. 1), the part corresponding to the neurite Nr and the part corresponding to the cell body So are not necessarily connected. One of the reasons for this is that when the input image is obtained by imaging, the linear portion such as the neurite Nr cannot be imaged with sufficient accuracy because it is thin. This tendency is also seen in images obtained by a fluorescence microscope that is often used for imaging cells and the like.

The probability distribution image Gp in FIG. 3 shows a plurality of pixel region fragments F corresponding to the neurite Nr. The pixel region fragment F1 and the pixel region fragment F2 are separated from each other by 2 pixels or more, and the pixel region fragment F2 and the pixel region fragment F3 are separated from each other by 2 pixels or more. In such a case, it is difficult to calculate the length of the neurite Nr when the neurite Nr is analyzed using the probability distribution image Gp, and it is not possible to appropriately acquire information from the neurite Nr.

The pixel area fragment F3 and the pixel area fragment F4 are adjacent to each other in the diagonal direction, but are not connected in the vertical direction or the horizontal direction. In such a case, it is not particularly limited whether the pixel region fragments F3 and F4 are connected or not connected, and in the analysis algorithm for analyzing the neurite Nr as described above. It can be appropriately set based on the definition of connectivity.

The data processing unit 51 calculates, from the probability distribution image Gp, a connected pixel area corresponding to the neurite Nr to be extracted and composed of a plurality of pixels that are integrally connected. An image showing a connected pixel area is called a connected area image.

FIG. 4 is a conceptual diagram showing a connected region image Gs obtained from the probability distribution image Gp of FIG. The connected pixel area Dc corresponds to the neurite Nr, and the pixels forming the connected pixel area Dc are connected to each other. Although the portion corresponding to the cell body So connected to the connected pixel area Dc is also shown in FIG. 4, this portion may or may not be shown in the connected area image Gs. In the following, the point that the data processing unit 51 sets the start point pixel and the end point pixel, derives the connected pixel area Dc that connects the start point pixel and the end point pixel, and generates the connected area image data corresponding to the connected area image Gs. Will be explained.

The data processing unit 51 sets a start point pixel and an end point pixel, and sequentially sets a plurality of pixels starting from the start point pixel. The set pixel is hereinafter referred to as a selected pixel. The data processing unit 51 extracts at least one of the selected pixels set when a condition based on the position of the end point pixel (hereinafter, referred to as an end condition) is satisfied. The pixel extracted here is called an extracted pixel.

The area setting unit 512 sets a search area that is a range for searching for extracted pixels. The search region is a range in which a candidate pixel designating unit 601 described later designates a candidate pixel. The area setting unit 512 sets the pixel area corresponding to the range set by the user in the probability distribution image Gp (FIG. 3) as the search area. The area setting unit 512 sets the entire probability distribution image Gp as the search area when there is no user setting.

FIG. 5 is a conceptual diagram showing a search area. The area setting unit 512 sets a rectangular range set by the user using a mouse cursor or the like as the search area D1.

The end point pixel setting unit 513 sets the end point pixel Pxt from the pixels forming the search area D1. The end point pixel Pxt may be one or plural. When the neurite Nr (FIG. 3) is the extraction target, the end point pixel Pxt is a pixel corresponding to the cell body So of the cell Ce (FIG. 1). The method of determining the pixel corresponding to the cell body So from the input image or the probability distribution image Gp is not particularly limited. The end point pixel setting unit 513 binarizes the probability distribution image Gp, deletes the neurite Nr by the opening process in the binarized image, and associates the pixel corresponding to the largest connected pixel region with the cell body So. It is possible to use a known method such as setting a pixel to be used.

The starting point pixel setting unit 514 sets the starting point pixel from the pixels forming the search area D1. The start point pixel setting unit 514 is a first binarized image data corresponding to a binarized image (hereinafter, referred to as a first binarized image) obtained by binarizing the brightness of each pixel in the probability distribution image Gp. Is generated, and the starting point pixel is set based on the first binarized image.

FIG. 6 is a diagram showing an example of the first binarized image Gb1 obtained by binarizing the probability distribution image Gp of FIG. In the first binarized image Gb1 of FIG. 6, the luminance values corresponding to the pixels constituting the pixel area fragments F (F1, F2, F3, and F4) of the probability distribution image Gp being black and the other portions being white. Has been converted to have. The threshold for binarization can be set appropriately based on the amount of noise in the probability distribution image Gp and the like. The cell body So corresponds to the end point pixel Pxt.

The start point pixel setting unit 514 is based on the distance from the end point pixel Pxt in the first binarized image Gb1 and the width of the pixel region corresponding to the extraction target in the first binarized image Gb1 and having the same value. Then, the starting point pixel Pxi is set. The starting point pixel setting unit 513 selects the pixel area fragment F1 having the largest number of pixels in the search area D1 among the pixel area fragments F, and sets the pixel farthest from the ending point pixel Pxt in the selected pixel area fragment F1 as the starting point pixel Pxi. It is preferable to set. As the distance between each pixel and the end point pixel Pxt in the pixel region fragment F1, the distance between each pixel and the center of gravity of the end point pixel Pxt, or the most relevant pixel among each pixel and the end point pixel Pxt. For example, the distance between adjacent pixels can be used.

The pixel extraction unit 600 (FIG. 2) sets the selected pixel in the search region D1 starting from the starting point pixel Pxi, and sets a pixel row formed by a plurality of selected pixels based on a predetermined condition as a plurality of extracted pixels. Extract. The starting point pixel Pxi may be included in the extracted pixels. This pixel extraction is performed by a method based on the Dijkstra method. In setting the selection pixel, the candidate pixel designating unit 601 designates a plurality of candidate pixels in the search region D1, and the selection pixel setting unit 602 sets a selection pixel from the plurality of candidate pixels.

FIG. 7 is a conceptual diagram for explaining setting of selected pixels by the pixel extracting unit 600. Pixels corresponding to a plurality of positions based on the position of the starting point pixel Pxi are designated as a plurality of candidate pixels Pxc. Here, the candidate pixel parameter is calculated for each of the candidate pixels Pxc corresponding to the starting point pixel Pxi. The candidate pixel parameter takes a non-negative value and corresponds to the distance in the Dijkstra method. If the starting point pixel Pxi and each candidate pixel Pxc are nodes, the candidate pixel parameter corresponds to each of the edges Ed connecting the nodes. Based on the value of the candidate pixel parameter, one pixel of the candidate pixels Pxc corresponding to the starting point pixel Pxi is selected and set as the selected pixel Pxd. The pixel set here becomes the selected pixel Pxd1 (arrow A3). The designation of the candidate pixel Pxc selected as the selected pixel Pxd is canceled as the candidate pixel.
It should be noted that every time the selected pixel Pxd is set, all the candidate pixels Pxc may be canceled once and the necessary entire candidate pixels may be reset again.

Next, in the search area D1 (FIG. 5) excluding the starting point pixel Pxi, pixels corresponding to a plurality of positions based on the position of the selected pixel Pxd1 are designated as a plurality of candidate pixels Pxc, respectively. Of the candidate pixels Pxc corresponding to the starting point pixel Pxi and the selected pixel Pxd1, the candidate pixel Pxc having the smallest sum of candidate pixel parameters corresponding to one or more edges Ed from the starting point pixel Pxi to each candidate pixel Pxc is new. It is set as the selected pixel Pxd2 (arrow A4). In this way, the plurality of selected pixels Pxd are sequentially set until the end condition is satisfied.

The designation of the candidate pixel Pxc by the candidate pixel designation unit 601 will be described in more detail. In the search area D1 excluding the start point pixel Pxi and the already set selection pixel Pxd, the candidate pixel designating unit 601 has a plurality of candidates at positions at least N pixels apart from each of the start point pixel Pxi and the already set selection pixel Pxd. The pixel Pxc is designated. Hereinafter, N is referred to as a candidate pixel position parameter. Here, when the pixel extracting unit 600 first extracts extracted pixels from the probability distribution image Gp, the pixel extracting unit 600 sets N to 2 or more.

Here, the distance between pixels when "X pixels are separated" is defined as follows. When pixels are arranged corresponding to the positions of grid points in the vertical and horizontal grids, the distance between two pixels vertically separated and the distance between two pixels horizontally separated are the unit grid in each direction. The width of is the distance one pixel away. That is, a pixel adjacent in the vertical direction or the horizontal direction to a certain pixel is a pixel separated by one pixel, and the distance between two pixels adjacent in the vertical direction or the horizontal direction is set as a distance separated by one pixel. .. The distance between two pixels that are diagonally separated is the longer of the vertical and horizontal distances.

For example, the pixel at the coordinate (0,0) and the pixel at the coordinate (1,1) are diagonally adjacent to each other, and the distance is 1 in the vertical direction and 1 in the horizontal direction. become. The pixel at the coordinate (0, 0) and the pixel at the coordinate (2, 2) are separated by 2 pixels because the distance 2 in the vertical direction and the distance 2 in the horizontal direction. The pixel at the coordinate (0,0) and the pixel at the coordinate (1,2) are separated by 2 pixels because the distance is 2 in the vertical direction and 1 is in the horizontal direction.

Since the candidate pixel position parameter N is 2 or more, and the candidate pixel Pxc is set at a distance of 2 pixels or more from the starting point pixel Pxi or the selected pixel Pxd, the number of times the selected pixel Pxd is set can be reduced, and the extraction target can be quickly extracted. Corresponding pixels can be extracted. Further, in the first binarized image Gb1 (FIG. 6), the pixel regions that are originally connected and correspond to the extraction target are interrupted at intervals of one pixel due to noise or the like (the probability that the extraction target corresponds to the low However, since it is possible to jump over the interval and set the pixel column including the selected pixel Pxd, it is possible to output the connected pixel region Dc that accurately reflects the extraction target.

FIG. 8 is a conceptual diagram showing designation of a candidate pixel Pxc corresponding to one selected pixel Pxd0 when the candidate pixel position parameter N is set to 6. In the example of FIG. 8, the candidate pixel Pxc is arranged at a position separated by 6 pixels in the vertical direction or the horizontal direction from the selected pixel Pxd0. Further, the candidate pixel Pxc is also arranged at a position moved by 6 pixels in both the vertical direction and the horizontal direction from the selected pixel Pxd0. The selected pixel Pxdp in the upper part of FIG. 8 is set as the selected pixel Pxd before the selected pixel Pxd0. A pixel that has already been set as the selected pixel Pxd is not designated as the candidate pixel Pxc. Similarly to the selected pixel Pxd, the candidate pixel Pxc is designated for the starting point pixel Pxi.

The selected pixel setting unit 602 sets a calculated pixel area Ds, which is a range of pixels using luminance values when calculating the candidate pixel parameters, based on the positions of the starting pixel Pxi or the selected pixel Pxd and the candidate pixel Pxc. .. Each pixel included in the calculated pixel area Ds is called a calculated pixel Pxs. The calculation pixel Pxs is preferably a pixel included in a predetermined range centered on a pixel (hereinafter, referred to as a central pixel Pxo) located on a line connecting the starting point pixel Pxi or the selected pixel Pxd and the candidate pixel Pxc. ..

In the example of FIG. 8, the central pixel Pxo is located on the line Lcd connecting the selected pixel Pxd0 and the candidate pixel Pxc1, and is the middle pixel of the seven pixels located on the line Lcd, in other words, the middle point of the line Lcd. It corresponds to the pixel including. In the example of FIG. 8, the calculated pixel area Ds is a 7×7 square area centered on the central pixel Pxo, and includes 49 calculated pixels Pxs. The center pixel Pxo and the calculated pixel area Ds are similarly set for the other candidate pixels Pxc.
The candidate pixel position parameter N and the calculated pixel area Ds may be set based on the width of the neurite Nr. The candidate pixel position parameter N and the calculated pixel region Ds can be set such that at least one of the widths of the candidate pixel position parameter N and the calculated pixel region Ds is larger than the width of the neurite Nr. For example, when the width of the neurite Nr is 2 pixels, the candidate pixel position parameter N can be set to a length of 5 pixels or 7 pixels. As another example, the candidate pixel position parameter N and the calculated pixel area Ds can be set such that the start point pixel Pxi or the selected pixel Pxd and the candidate pixel Pxc are included in the calculated pixel area Ds of the candidate pixel Pxc. .. As a result, it is possible to use more pixels of the probability distribution image Gp and reduce the waste of information for calculation.

The selected pixel setting unit 602 calculates the candidate pixel parameter d from the brightness values of the plurality of calculated pixels Pxs included in the calculated pixel area Ds corresponding to each candidate pixel Pxc. The candidate pixel parameter d is calculated using the sum of the brightness values of the calculation pixels Pxs included in the calculation pixel area Ds or the average value such as the arithmetic mean. For example, the candidate pixel parameter d can be calculated by the following equation (1) using I _patch and a as a constant, which is the sum of the luminances included in the calculated pixel region Ds corresponding to each candidate pixel Pxc.
d = exp(-I _patch /a) (1)

FIG. 9 shows the value of the candidate pixel parameter d with respect to the sum I _patch of the brightness values of the calculated pixel Pxs calculated by the equation (1). The candidate pixel parameter d is set to monotonically decrease as the sum I _patch increases. Since the luminance value of the probability distribution image Gp (FIG. 8) indicates the probability that the pixel corresponds to the extraction target, the higher the luminance value of the calculated pixel Pxs, the smaller the candidate pixel parameter d. This corresponds to a small distance value in the Dijkstra method.

The selected pixel setting unit 602 calculates the sum of the candidate pixel parameters d corresponding to each edge Ed (FIG. 7) from the start point pixel Pxi to the candidate pixel Pxc among the plurality of candidate pixels Pxc corresponding to the start point pixel Pxi and the selected pixel Pxd. The candidate pixel Pxc having the smallest value of is set as a new selected pixel Pxd. When the new selected pixel Pxd is set, the candidate pixel designating unit 601 adds the candidate pixel Pxc that has already been designated as described above to the candidate pixel at each of a plurality of positions based on the position of the new selected pixel Pxd. Pxc is designated, and the selected pixel setting unit 602 further sets a new selected pixel Pxd from the plurality of candidate pixels Pxc. The pixel extraction unit 600 sets a plurality of selected pixels Pxd by the process of repeating the designation of the candidate pixel Pxc and the setting of the selected pixel Pxd (hereinafter referred to as the selected pixel setting process).

The pixel extraction unit 600 sets a termination condition based on the end point pixel Pxt (FIG. 5) before performing the selection pixel setting process, and terminates the selection pixel setting process when the termination condition is satisfied. At least one of the end conditions (hereinafter, referred to as a first end condition) is a case where the candidate pixel Pxc (FIG. 8) set by the candidate pixel designation unit 601 overlaps with the end point pixel Pxt. When the first termination condition is satisfied, the pixel extraction unit 600 terminates the selection pixel setting process, and selects from among the plurality of selection pixels Pxd set at that point, the number of pixels that is the smallest number of pixels away from the end point pixel Pxt. The plurality of selected pixels Pxd included in the pixel column corresponding to the pixel Pxd are stored in the storage unit 43 as extraction pixels. Here, the pixel row is composed of the start point pixel Pxi and at least a part of the plurality of selected pixels Pxd selected in the selection pixel setting process, and is separated from the start point pixel Pxi and the end point pixel Pxt by the smallest number of pixels. It is a pixel column that connects the selected pixels Pxd. At least a part of the plurality of selected pixels Pxd is a plurality of selected pixels Pxd forming a column having the smallest sum of candidate pixel parameters d among a plurality of columns of the plurality of selected pixels Pxd starting from the start point pixel Pxi, These selected pixels Pxd are extraction pixels. Here, “connecting” means that adjacent pixels in a pixel row are associated with each other. The “correspondence” between the pixel A and the pixel B means that the pixel B is arranged at any position where the candidate pixel Pxc can be arranged with respect to the pixel A. After that, the pixel extraction unit 600 discards the set plurality of selected pixels Pxd and returns to the state in which the selected pixels Pxd are not set. Then, as described later, the area setting unit 512 sets the connected pixel area Dc based on the plurality of extracted pixels.
It should be noted that the selection pixel Pxd that has been set may be discarded at any time as long as it is performed before a new selection pixel setting process is started.

Another one of the end conditions (hereinafter referred to as the second end condition) is that the selected pixel Pxd is set a predetermined number of times. The number of times can be appropriately set based on the magnification and resolution of the probability distribution image Gp (FIG. 8), the characteristics of the extraction target, and the like. When the second termination condition is satisfied, the pixel extraction unit 600 discards the set plurality of selected pixels Pxd and returns to the state where the selected pixel Pxd is not set. After that, it is preferable to change the condition such as changing the position of the starting point pixel Pxi or changing the candidate pixel position parameter N, and to redo the selected pixel setting process. By setting the second end condition, unnecessary search is avoided when the pixel region fragment F including the starting point pixel Pxi and the below-described candidate region Fc are not the neurite Nr to be extracted, and the calculation cost is reduced. Can be reduced.

The area setting unit 512 sets the connected pixel area Dc (FIG. 4) based on the extracted plurality of extracted pixels. The area setting unit 512 associates each of the starting point pixel Pxi and the extracted plurality of extracted pixels with a pixel area in a predetermined range (hereinafter, referred to as a connected element pixel area). Each of the connected element pixel regions corresponding to each of the start point pixel Pxi and the plurality of extracted pixels has the predetermined range set in advance so that a part of the pixels overlaps the other connected element pixel regions. The area setting unit 512 sets a pixel area including a pixel corresponding to at least one of the set plurality of connected element pixel areas as a connected pixel area Dc.

FIG. 10 is a conceptual diagram for explaining the setting of the connected pixel area Dc. In FIG. 10, the connected element pixel region De is set as a 7×7 square pixel region centered on the start point pixel Pxi or the extracted pixel Pxe. In FIG. 10, the candidate pixel position parameter N is set to 3 and the selected pixel setting process is performed. Since the width of the connected element pixel area De is sufficiently larger than the candidate pixel position parameter N, each connected element pixel area De has an overlapping portion with another connected element pixel area De, and a connected connected pixel area Dc is obtained. ing. As an example, it is shown that the connected element pixel area De1 corresponding to the starting point pixel Pxi and the connected element pixel area De2 corresponding to the extracted pixel Pxe1 overlap for some pixels.
The shape and size of the connected element pixel area De are not particularly limited. The shape or size of the connected element pixel area De may be determined based on the shape or size of the calculated pixel area Ds (FIG. 8). For example, the shape and size of the connected element pixel area De can be made equal to the shape and size of the calculated pixel area Ds. As a result, the size of the pixel region that contributes when calculating the candidate pixel parameter d and the size of the connected element pixel region De become substantially equal, so that the connected pixel region Dc corresponding to the extraction target can be set more accurately. it can. In addition, the size of the connected element pixel region De may be set to a size that includes the starting point pixel Pxi and the extraction pixel Pxe adjacent to each other or the two extraction pixels Pxe adjacent to each other.

The image generation unit 700 generates image data (hereinafter, referred to as connected area image data) corresponding to the connected area image Gs (see FIG. 4) indicating the connected pixel area Dc. In the connected region image Gs, the portion corresponding to the connected pixel region Dc is shown separately from other portions. The image generation unit 700 sets a portion corresponding to the connected pixel area Dc in the connected area image Gs or a portion obtained by adding the end point pixel Pxt (FIG. 6) to the portion to at least one of the hue, the brightness, and the saturation. Connected area image data is generated in a different manner. The generated connected area image data becomes output image data from the data processing unit 51. The connected region image data is then appropriately subjected to morphological analysis and the like, and the neurite length is calculated and the like.

Returning to FIG. 1, the output control unit 52 of the control unit 50 controls the output unit 44 to output the connected region image Gs as an output image.

The device control unit 53 of the control unit 50 controls each unit of the culture unit 100 based on the input from the input unit 41 (arrow A2). The device control unit 53 controls culture and controls the imaging unit 20 to perform imaging.

FIG. 11 is a flowchart showing the flow of the image generation method according to this embodiment. In step S1001, the culture unit 100 cultures the cells Ce. When step S1001 ends, step S1003 starts. In step S1003, the imaging unit 20 images the cultured cells Ce, and the information processing unit 40 acquires input image data corresponding to the captured image. When step S1003 ends, step S1005 starts.

In step S1005, the probability distribution image generation unit 511 converts the input image data into probability distribution image data corresponding to the probability distribution image Gp. When step S1005 ends, step S1007 starts. In step S1007, the end point pixel setting unit 513 sets the end point pixel Pxt. When step S1007 ends, step S1009 starts.

In step S1009, the starting point pixel setting unit 514 sets the starting point pixel Pxi. When step S1009 ends, step S1011 starts. In step S1011, the pixel extraction unit 600 sets a plurality of selected pixels Pxd and extracts a plurality of extracted pixels. When step S1011 ends, step S1013 starts.

In step S1013, the region setting unit 512 sets the connected pixel region Dc based on the extracted plurality of extracted pixels Pxe. When step S1013 ends, step S1015 starts. In step S1015, the image generation unit 700 generates connected region image data corresponding to the connected region image Gs indicating the connected pixel region Dc. When step S1015 ends, step S1017 starts.

In step S1017, the output unit 44 outputs the connected area image Gs. When step S1017 ends, the process ends.

FIG. 12 is a flowchart showing the flow of step S1011 in the flowchart of FIG. When step S1009 ends, step S111 starts. In step S111, the candidate pixel designating unit 601 designates a plurality of pixels located at least N pixels from the starting point pixel Pxi as candidate pixels Pxc. When step S111 ends, step S113 starts. In step S113, the selected pixel setting unit 602 calculates the candidate pixel parameter d from the brightness of the plurality of calculated pixels Pxs included in the calculated pixel area Ds corresponding to each candidate pixel Pxc. When step S113 ends, step S115 starts.

In step S115, the selected pixel setting unit 602 sets the selected pixel Pxd from the plurality of candidate pixels Pxc based on the candidate pixel parameter d of each candidate pixel Pxc. When step S115 ends, step S117 starts. In step S117, the candidate pixel designating unit 601 designates, as candidate pixels Pxc, a plurality of pixels existing at positions at least N pixels apart from the already set selected pixel Pxd. When step S117 ends, step S119 starts.

In step S119, the candidate pixel designation unit 601 determines whether at least one of the plurality of candidate pixels Pxc is the end point pixel Pxt. When the plurality of candidate pixels Pxc include the end point pixel Pxt, the candidate pixel designation unit 601 makes an affirmative decision in step S119, and step S127 is started. If the plurality of candidate pixels Pxc does not include the end point pixel Pxt, the candidate pixel designation unit 601 makes a negative determination in step S119 and starts step S121.

In step S121, the selected pixel setting unit 602 calculates the candidate pixel parameter d from the brightness of the plurality of calculated pixels Pxs included in the calculated pixel area Ds corresponding to each candidate pixel Pxc. When step S121 ends, step S123 starts. In step S123, the selected pixel setting unit 602 sets a new selected pixel Pxd from the plurality of candidate pixels Pxc based on the candidate pixel parameter d of each candidate pixel Pxc. When step S123 ends, step S125 starts.

In step S125, the pixel extraction unit 600 determines whether or not the selected pixel Pxd has been set a predetermined number of times. When the selected pixel Pxd has been set the number of times described above, the pixel extraction unit 600 makes an affirmative decision in step S125, and the process ends. If the number of times the selected pixel Pxd is set is less than the above number, the pixel extraction unit 600 makes a negative decision in step S125 and returns to step S117.

In step S127, the pixel extraction unit 600 stores the extracted pixel including at least one of the set selected pixels Pxd in the storage unit 43, and then sets the selected pixel Pxd to the unset state. When step S127 ends, step S1013 (FIG. 11) starts.
It should be noted that at the stage where the candidate pixel Pxc is designated, the determination in step S119 is not performed, but it is determined whether or not the end pixel Pxt matches after the selected pixel Pxd is set. If they match, step S127 is started. It may be one.

According to the above-mentioned embodiment, the following effects can be obtained.
(1) In the image generation device 1 or the image generation method of the present embodiment, the starting point pixel setting unit 514 that sets the starting point pixel Pxi from the plurality of pixels that form the probability distribution image Gp, and the plurality of pixels that form the probability distribution image Gp. A pixel extraction unit 600 that specifies a plurality of candidate pixels Pxc from a plurality of pixels other than the start point pixel Pxi and sets a selected pixel Pxd from the plurality of candidate pixels Pxc. The selection pixel setting process of designating a plurality of candidate pixels Pxc from a plurality of pixels excluding the set selection pixel Pxd among the plurality of pixels forming the image Gp and newly setting a selection pixel Pxd from the plurality of candidate pixels Pxc is repeated. Then, the plurality of candidate pixels Pxc are separated from the exclusion pixel which is the start point pixel Pxi and the selection pixel Pxd that are excluded at the time of designation by at least two pixels. With this, it is possible to realize the process of quickly extracting the pixel corresponding to the extraction target while suppressing the deterioration of accuracy.

(2) The image generation device 1 of the present embodiment includes the probability distribution image generation unit 511 that acquires the probability distribution image data corresponding to the probability distribution image Gp, and the pixel extraction unit 600 includes the excluded pixel and the candidate pixel Pxc. At least a part of the plurality of selection pixels Pxd obtained by the selection pixel setting process by setting a new selection pixel Pxd from the plurality of candidate pixels Pxc based on the brightness of the plurality of calculation pixels Pxs set based on the position. Is extracted as an extraction pixel Pxe, and an image generation unit 700 that generates connection region image data corresponding to the connection region image Gs based on the extracted plurality of extraction pixels Pxe. With this, it is possible to quickly extract the pixel corresponding to the extraction target and suppress the deterioration of accuracy, and provide the image based on the extraction.

(3) In the image generation device 1 of the present embodiment, the probability distribution image Gp includes the region setting unit 512 that sets the connected pixel region Dc including at least a plurality of extracted pixels Pxe, and the image generation unit 700 includes the connected pixel region. Connected area image data corresponding to the connected area image Gs indicating Dc is generated. Thereby, the connected pixel area (connected pixel area Dc) can be obtained, and morphological analysis and the like can be performed accurately.

(4) The image generation apparatus 1 of the present embodiment includes the end point pixel setting unit 513 that sets one or more end point pixels Pxt connected to the connected pixel region Dc, and the pixel extraction unit 600 sets the end point pixel Pxt at the position. Based on this, a condition for ending the selected pixel setting process is set. Accordingly, when there is a portion to which the extraction target is connected in the input image, the connected pixel area Dc that reflects the connection can be set.

(5) In the image generation device 1 of the present embodiment, the pixel extraction unit 600 determines whether the candidate pixel Pxc designated by the candidate pixel setting unit 601 corresponds to at least one of the end point pixels Pxt, or a plurality of candidate pixels Pxc. When the setting of the selected pixel Pxd from is performed a predetermined number of times, it is determined that the end condition is satisfied, and the selected pixel setting process is ended. As a result, the connected pixel region Dc is properly connected to the portion of the input image to which the extraction target is connected, and unnecessary search is avoided when the selected pixel Pxd that does not correspond to the extraction target is erroneously set. The calculation cost can be reduced.

(6) In the image generation device 1 of the present embodiment, the selected pixel setting unit 602 calculates the candidate pixel parameter d for each candidate pixel Pxc based on the brightness of the plurality of calculated pixels Pxs, and the candidate pixel parameter d. The selected pixel Pxd is set from the plurality of candidate pixels Pxc based on the above. Thereby, the pixel corresponding to the extraction target can be accurately extracted.

(7) In the image generation device 1 of the present embodiment, the selected pixel setting unit 602 calculates the candidate pixel parameter d based on the sum of the brightness of the plurality of calculated pixels Pxs or the arithmetic mean. As a result, noise can be reduced, and the pixel corresponding to the extraction target can be accurately extracted.

(8) In the image generation device 1 of the present embodiment, the plurality of calculation pixels Pxs are a plurality of pixels included in the calculation pixel region Ds based on the selected pixel Pxd and the candidate pixel Pxc. Thereby, the pixel corresponding to the extraction target can be accurately extracted based on the brightness of the pixel at the appropriate position.

(9) In the image generation device 1 of the present embodiment, the range of the calculated pixel region Ds is a range centered on the central pixel Pxo located on the line Lcd connecting the selected pixel Pxd and the candidate pixel Pxc. As a result, the pixel corresponding to the extraction target can be more accurately extracted based on the brightness of the pixel at the more appropriate position.

(10) In the image generation device 1 of the present embodiment, the area setting unit 512 sets the connected pixel area Dc from the plurality of extracted pixels Pxe based on the preset range or the calculated pixel area Ds. Thereby, the connected pixel area Dc corresponding to the extraction target can be accurately set.

(11) In the image generation device 1 of the present embodiment, the probability distribution image Gp calculates the probability corresponding to the extraction target for each pixel of the input image obtained by imaging using learned machine learning, It is an image in which luminance is associated based on probability. As a result, the probability distribution can be accurately obtained by using the image processing algorithm that has been learned according to the characteristics of the extraction target.

(12) In the image generation device 1 of the present embodiment, the machine learning inputs a plurality of captured images obtained by imaging and an image showing a portion corresponding to an extraction target in each captured image to the arithmetic device. It is deep learning that was learned by. As a result, the distribution of the above probabilities can be obtained accurately based on deep learning.

(13) In the image generating apparatus 1 of the present embodiment, the user of the image generating apparatus 1 can specify the search area D1 that is the range in which the starting point pixel Pxi or the candidate pixel Pxc is arranged in the probability distribution image Gp. This makes it possible to appropriately specify the range for extracting the fixed pixel Pxd based on the user's instruction.

(14) In the image generation device 1 of the present embodiment, the starting point pixel setting unit 514 uses the starting point pixel using the data corresponding to the first binarized image Gb1 obtained by binarizing the luminance in the probability distribution image data. Set Pxi. As a result, the starting point pixel Pxi can be appropriately set in the portion corresponding to the extraction target.

(15) In the image generation device 1 of the present embodiment, the start point pixel setting unit 514 sets the distance from the end point pixel Pxt in the first binarized image Gb1 and the same value in the first binarized image Gb1. The start point pixel Pxi is set based on the size of the connected pixel area fragment F. This makes it possible to appropriately set the starting point pixel Pxi based on the distance from the portion (cell body So or the like) connected to the extraction target in the input image while reducing the influence of noise.

(16) In the image generation device 1 of the present embodiment, the extraction target is the portion corresponding to the neurite Nr in the probability distribution image Gp. As a result, the pixel corresponding to the neurite Nr can be quickly extracted while suppressing deterioration in accuracy, and image data for accurately analyzing the length of the neurite Nr and the like can be provided. If the shape including the length of the neurite Nr can be analyzed, it becomes possible to quantify the effect of a drug and the degree of disease progression by examining the change in a sample from a living body or an experiment in which a drug is administered to nerve cells. ..

(17) In the image generation device 1 of the present embodiment, the end point pixel Pxt corresponds to the cell body So connected to the neurite Nr. As a result, it is possible to set the connected pixel area Dc that reflects the connection to the cell body So.

(18) The image generation device 1 of the present embodiment includes the output unit 44 that outputs the connected region image Gs. This allows the user to visually recognize the connected region image Gs.

(19) The imaging device according to the present embodiment includes an information processing unit 40 as an image generation device and an imaging unit 20. This makes it possible to quickly extract the pixel corresponding to the extraction target from the captured image while suppressing deterioration in accuracy.

(20) The culture device according to the present embodiment includes an information processing unit 40 as an image generation device and a culture unit 100 that cultures the cells Ce. As a result, the cultured cells Ce can be analyzed using the connected region image Gs, and the culture conditions and the culture time can be adjusted based on this analysis.

The following modifications are also within the scope of the present invention, and can be combined with the above-described embodiment.
(Modification 1)
In the above-described embodiment, the determined connected pixel area Dc may be set as the search area D1 and the definite pixel may be extracted again.

FIG. 13 is a conceptual diagram for explaining the pixel extraction method of this modification. The area setting unit 512 sets the connected pixel area Dc (FIG. 10) set in the above embodiment as the search area D1. The candidate pixel designating unit 601 designates the candidate pixel Pxc (FIG. 8) in the search region D1 thus set, and the pixel extracting unit 600 re-extracts the extracted pixel Pxe. In the setting of the selected pixel Pxd (FIG. 8) in this second extraction, the candidate pixel position parameter N is set to a value smaller than the candidate pixel position parameter N (N is 2 or more) in the first extraction of the above-described embodiment. Therefore, the candidate pixel designation unit 601 sets the candidate pixel position parameter N to 1 or more in this second extraction. Although the starting point pixel setting unit 514 sets the starting point pixel Pxi in the search area D1, the starting point pixel Pxi in the above-described embodiment may be used as it is.

FIG. 13 shows the extracted pixel Pxe extracted when the candidate pixel position parameter N is set to 1 in the second extraction. The area setting unit 512 sets a connected element pixel area De (FIG. 10) to each of the starting point pixel Pxi and the extracted pixel Pxe extracted in the extraction again, and a new connected element pixel area De is connected. A connected pixel area can be set. In this case, the width of the connected element pixel region De is set to be smaller than the width of the connected element pixel region De in the first extraction of the above-described embodiment, and for example, 7 pixels are set when the first connected pixel area Dc is set. At the time of setting again, it can be set to 2 pixels or 3 pixels.

As described above, the setting of the first connected pixel region Dc is extracted by setting the width of the candidate pixel position parameter N and the width of the connected element pixel region De to be smaller than that in the extraction of the first extracted pixel Pxe in the second extraction. The pixels corresponding to the target can be roughly extracted, and the second extraction can be performed with higher accuracy. In the second extraction, even if the accuracy is higher than that in the first extraction, the search area D1 is narrowed, so that the increase in the amount of calculation can be suppressed.

FIG. 14 is a flowchart showing the flow of the image generation method according to this modification. Since steps S2001 to S2013 correspond to steps S1001 to S1013 of the flowchart of FIG. 11, respectively, description thereof will be omitted. When step S2013 ends, step S2015 starts.

In step S2015, the area setting unit 512 sets the connected pixel area Dc set in step S2013 as the search area D1, and the pixel extraction unit 600 sets the plurality of selected pixels Pxd again and the plurality of extracted pixels Pxe. To extract. When step S2015 ends, step S2017 starts. In step S2017, the area setting unit 512 sets the connected pixel area Dc based on the plurality of extracted pixels Pxe extracted again. When step S2017 ends, step S2019 starts. Steps S2019 and S2021 correspond to steps S1015 and S1017 of the flowchart of FIG. 11, respectively, and thus description thereof will be omitted.

In the image generation apparatus of this modification, the starting point pixel setting unit 514 sets the starting point pixel Pxi in the connected pixel area Dc (search area D1) set by the area setting unit 512, and the pixel extraction unit 600 sets the connected pixel area. A plurality of extracted pixels Pxe are extracted again from Dc, and the candidate pixel designating section 514 sets candidate pixels Pxc at a plurality of positions apart from the starting point pixel Pxi and the selected pixel Pxd by one or more pixels in this second extraction. The area setting unit 512 sets the connected pixel area Dc including the pixels extracted by the second extraction. As a result, it is possible to suppress an increase in calculation amount while extracting pixels with higher accuracy.

(Modification 2)
In the above-described embodiment, the shape or size of the connected element pixel area De may be determined based on the value of the candidate pixel parameter d when the calculated pixel area Ds (FIG. 8) is changed. For example, it is assumed that the candidate pixel parameter d is based on the average value of the brightness values of the calculated pixels Pxs in the calculated pixel area Ds. In this case, when the width of the neurite Nr to be extracted is smaller than the calculated pixel area Ds, the ratio of the calculated pixel Pxs that does not correspond to the neurite Nr in the calculated pixel area Ds increases and the candidate pixel parameter d changes.

FIG. 15 is a conceptual diagram showing the correspondence between the calculated pixel area Ds and the neurite Nr. Since the calculated pixel area Ds is larger than the width W of the neurite Nr, the number of calculated pixels Pxs2 that do not overlap the neurite Nr is larger than that of the calculated pixels Pxs1 that overlap the neurite Nr. When the calculated pixel area Ds is reduced from the size shown in FIG. 15 and the width of the calculated pixel area Ds becomes approximately the same as the width W of the neurite Nr, even if the calculated pixel area Ds is made smaller thereafter, The change in the candidate pixel parameter d is smaller than before. Therefore, when the calculated pixel area Ds is changed to calculate the candidate pixel parameter d, the calculated pixel area Ds when the change of the candidate pixel parameter d becomes small is set as the selected pixel Pxd (FIG. 8). When the corresponding connected element pixel area De is set, the connected pixel area Dc (FIG. 4) based on the width of the neurite Nr can be generated.

The selected pixel setting unit 602 calculates a plurality of candidate pixel parameters d by changing the size of the calculated pixel area Ds for each candidate pixel Pxc (FIG. 8). The area setting unit 512 can set the size of the connected element pixel area De when the candidate pixel becomes the extracted pixel Pxe based on the calculated candidate pixel parameter d.

In the image generation apparatus of this modification, the area setting unit 512 can change the size of the connected pixel area Dc based on the candidate pixel parameter d when the calculated pixel area Ds is changed. Accordingly, it is possible to provide the connected pixel region Dc based on the width of the extraction target linear portion and the like.

(Modification 3)
Although the subject of the input image is the cell Ce (FIG. 1) and the linear part of the extraction target is the neurite Nr in the above-described embodiment, the extraction target is limited to this example as long as the linear part is included. Instead, it is also preferable to use a blood vessel, for example.

Furthermore, it is more preferable that the subject of the input image is the fundus and the linear portion to be extracted is the blood vessel at the fundus. The fundus image shows that blood vessels extend in various directions from the optic disc. Therefore, the linear portion to be extracted is a blood vessel, and the probability distribution image generation unit 511 generates a probability distribution image Gp (FIG. 5) in which the probability that each pixel corresponds to a blood vessel is associated with a brightness value from the input image. .. The end point pixel setting unit 513 extracts the pixel area corresponding to the optic disc and sets it as the end point pixel Pxt. The starting point pixel setting unit 514 sets the starting point pixel Pxi (FIG. 6) to the pixel region fragment F in the first binarized image. The pixel extraction unit 600 extracts a plurality of extracted pixels Pxe corresponding to blood vessels, and the region setting unit 512 sets the connected pixel region Dc.

In the image generation device 1 of this modification, the extraction target is the portion corresponding to the blood vessel in the probability distribution image Gp. This makes it possible to quickly extract a portion corresponding to a blood vessel from an image of a blood vessel as a subject while suppressing deterioration in accuracy. The length and area of the blood vessel can be calculated based on the obtained information on the portion corresponding to the blood vessel, and the blood vessel can be quantified. Although there may be variations in the evaluation of the effect of treatment and the degree of disease progression by a doctor's visual inspection, the method of this modification allows objective evaluation.

In the image generation device 1 of the present modification, the extraction target is a portion corresponding to a blood vessel in the probability distribution image Gp, the probability distribution image Gp is an image corresponding to the fundus, and the end point pixel Pxt corresponds to the optic disc. .. This makes it possible to quickly extract blood vessels in the fundus of the eye while suppressing deterioration in accuracy.

-Second Embodiment-
The image generation device 2 of the second embodiment has the same configuration as the image generation device 1 according to the first embodiment, but the configuration of the data processing unit is different from that of the first embodiment. In the second embodiment, a method for accurately extracting pixels corresponding to a plurality of neurites Nr in the probability distribution image Gp will be described. The same parts as those of the first embodiment are referred to by the same reference numerals as those of the first embodiment, and the description thereof will be omitted depending on the case.

FIG. 16 is a conceptual diagram showing the configuration of the image generating apparatus 2 of this embodiment. The image generation device 2 includes components other than the data processing unit 51a in the same manner as in FIG. 1, but the illustration is omitted. The image generation device 2 differs from the image generation device 1 of the above-described embodiment in that it includes a reliability calculation unit 515, a determination unit 516, and a candidate/determined region setting unit 517.

In the following, when the search region D1 (FIG. 5) includes an image portion corresponding to a plurality of branched neurites Nr, or includes an image portion corresponding to a plurality of neurites Nr extending from one cell body So. Extraction of pixels corresponding to these neurites Nr in the case will be described. The cell-cell interaction can be quantified from the positional information of the plurality of neurites Nr or nerve cells obtained in the present embodiment.

The reliability calculation unit 515 calculates the reliability of whether or not the plurality of extracted pixels Pxe extracted by the pixel extraction unit 600 correspond to the portion corresponding to the extraction target. The reliability calculation unit 515 calculates the reliability parameter R indicating this reliability based on the candidate pixel parameter d corresponding to the selected pixel Pxd when the selected pixel Pxd corresponding to the extracted pixel Pxe is set. The reliability calculation unit 515 calculates the arithmetic mean of the candidate pixel parameters d corresponding to the selected pixel Pxd when the selected pixel Pxd corresponding to the extracted pixel Pxd is set for the extracted pixel row of the extracted pixels Pxd. It is preferable to calculate an average value d _{ave of the above} as the reliability parameter R. Here, it is assumed that the smaller the numerical value of the reliability parameter R is, the higher the reliability is. However, it is not particularly limited as long as the correspondence can be established, and it is appropriately set based on the definition of the reliability parameter R.

The determination unit 516 determines whether or not the plurality of extracted pixels Pxe extracted by the pixel extraction unit 600 correspond to one continuous neurite Nr based on the reliability parameter R (hereinafter, reliability determination Call). The determination unit 516 performs the reliability determination based on whether the reliability parameter R satisfies the condition based on the threshold Th. The threshold value Th is set in advance based on an example in which the extraction pixel Pxe has been extracted in the past, and is stored in the storage unit 43. For example, assume that the reliability calculation unit 515 calculates d _ave as the reliability parameter R under the condition that the candidate pixel Pxc with the smallest candidate pixel parameter d is set as the selected pixel Pxd. In this case, when the reliability parameter R is larger than the threshold Th, the determination unit 516 determines that the extracted pixels Pxd extracted by the pixel extraction unit 600 have insufficient reliability, and discards them. The connected pixel area Dc is not set for the extracted pixel Pxe that is the target of the above. The reliability determination is preferably performed when the connected pixel region Dc connected to the end point pixel Pxt is set, for example, when the selected pixel setting process ends due to the first end condition described above.

As described above, it is possible to reduce the risk of extracting an image portion that does not correspond to the neurite Nr as the neurite Nr by performing the determination based on the reliability of the extracted pixel Pxe.

The candidate/determined area setting unit 517 is an area that is a candidate for a portion corresponding to the neurite Nr in the probability distribution image Gp (hereinafter, referred to as a candidate area) before the selection pixel setting processing by the pixel extraction unit 600 is performed. To set. The candidate/determined region setting unit 517 performs the second binarization corresponding to the binarized image (hereinafter, referred to as the second binarized image) obtained by binarizing the brightness of each pixel in the probability distribution image Gp. Generate image data. The search region D1 is divided into a region having a high probability of corresponding to the extraction target and a region having a low probability based on the threshold value at the time of binarization.
The first binarized image may be used as the second binarized image.

The candidate/determined area setting unit 517 sets a candidate area based on the second binarized image data. The candidate/determined area setting unit 517 sets an area having a high probability of corresponding to the extraction target as a candidate area. Further, the candidate/determined area setting unit 517 sets the confirmed area. The definite area is assumed to have no corresponding area until the extraction pixel Pxe is extracted by the pixel extraction unit 600. As a result of extraction of the extracted pixels Pxe by the pixel extraction unit 600, when it is determined that at least a part of the candidate regions is a region corresponding to the extraction target, the candidate/determined region setting unit 517 selects the determined candidate regions as candidates. Remove it from the area and set it as the fixed area.

The candidate/determined area setting unit 517 generates a candidate area image data corresponding to the candidate area image indicating the candidate area and a confirmed area image data corresponding to the confirmed area image indicating the confirmed area, and It functions as a fixed area image generation unit. The modes of the candidate area image and the finalized area image are not particularly limited, but a binarized image is preferable from the viewpoint of saving the amount of information.

FIG. 17A is a diagram showing an example of the candidate area image Gc and the finalized area image Gd before the pixel extraction unit 600 performs the selected pixel setting process. In the candidate area image Gc, four separated candidate areas Fc1, Fc2, Fc3 and Fc4 (hereinafter, the candidate areas are generally indicated by the symbol Fc) corresponding to the same branched neurite Nr (FIG. 4) are shown. Has been done. On the other hand, in the fixed area image Gd, the fixed area is not set at this stage. In FIG. 17A, the position of the cell body So to which the neurite Nr to be extracted is connected is schematically shown to the left of the candidate region image Gc and the confirmed region image Gd (FIGS. 17C and 17C). The same applies to D)).

FIG. 17B shows a connected pixel connected from the candidate region Fc2 to the cell body So through the candidate region Fc1 by extraction of the extracted pixel Pxe by the pixel extraction unit 600 and setting of the connected pixel region Dc by the region setting unit 512. It is a conceptual diagram which shows the point where the area|region Dc12 was set. The candidate areas Fc3 and Fc4 are separated from the connected pixel area Dc12.

FIG. 17C is a conceptual diagram showing updating of the candidate area Fc and the finalized area. It is assumed that the determination unit 516 determines the reliability of the extracted pixel Pxe corresponding to the connected pixel region Dc12 and determines the reliability as high. In this case, the candidate/determined area setting unit 517 updates the settings of the candidate areas Fc1 and Fc2 as those that are not the candidate area Fc, and the connected pixel area Dc12 is defined area Fd1 (hereinafter, the defined area is generally indicated by the symbol Fd). ).

When some candidate regions Fc are set as the finalized regions Fd, the starting point pixel setting unit 514 sets the starting point pixels Pxi in any of the remaining candidate regions Fc. The end point pixel setting unit 513 sets the cell body So and the set confirmed region Fd1 as the end point pixel Pxt.

FIG. 17D is a conceptual diagram showing the candidate area image Gc and the finalized area image Gd after the candidate area Fc and the finalized area Fd are further updated from the state of FIG. 17C. From the state of FIG. 17C, the starting point pixel Pxi is set in the candidate region Fc4, the cell body So and the connected pixel region Dc12 are set as the end pixel Pxt, and the extraction pixel Pxe is extracted and the connected pixel region Dc3 is set. It was conducted. Further, it is assumed that the connected pixel region Dc3 is determined to have high reliability by reliability determination. At this time, the candidate/determined area setting unit 517 updates the setting so that the candidate area Fc4 is not the candidate area Fc, and sets a new confirmed area Fd2 in which the confirmed area Fd1 and the connected pixel area Dc3 are connected to the confirmed area Fd. To do.

FIG. 18 is a flowchart showing the flow of the image generation method according to this modification. Steps S3001 to S3005 correspond to steps S1001 to S1005 of the flowchart of FIG. 11, respectively, and thus description thereof will be omitted. When step S3005 ends, step S3007 starts.

In step S3007, the candidate/determined area setting unit 517 generates data corresponding to the candidate area Fc and the confirmed area Fd. When step S3007 ends, step S3009 starts. In step S3009, the start point pixel setting unit 514 sets the start point pixel Pxi, the end point pixel setting unit 513 sets the end point pixel Pxt, and the pixel extraction unit 600 sets the plurality of selected pixels Pxd and sets the plurality of extracted pixels Pxe. And the area setting unit 512 sets the connected pixel area Dc based on the extracted pixel Pxe. When step S3009 ends, step S3011 starts.

In step S3011, the determination unit 516 determines the reliability of the plurality of extracted pixels Pxe extracted in step S3009. When step S3011 ends, step S3013 starts. In step S3013, the candidate/determined area setting unit 517 updates the data corresponding to the candidate area Fc and the confirmed area Fd based on the result of the reliability determination. When the reliability is determined to be high, the candidate area Fc in which the plurality of extracted pixels Pxe subjected to the reliability determination are arranged is deleted from the candidate area, and the connected pixel set in step S3009 is set. The area Dc is set as the finalized area Fd. When the reliability is determined to be low, the extracted pixel Pxe extracted in step S3009 and the set connected pixel area Dc are discarded. Further, in this case, the candidate region Fc in which the starting point pixel Pxi is arranged or the plurality of extracted pixels Pxe are arranged in the selected pixel setting process when a plurality of extracted pixels Pxe subjected to the reliability determination are extracted. The candidate area Fc can be deleted from the candidate area Fc. When step S3013 ends, step S3015 starts.

In step S3015, the candidate/determined area setting unit 517 determines whether the candidate area Fc still remains. If the candidate area Fc still remains, the candidate/determined area setting unit 517 makes an affirmative decision in step S3015 and returns to step S3009. If no candidate region Fc remains, the candidate/determined region setting unit 517 makes a negative determination in step S3015 and starts step S3017.

Steps S3017 and S3019 are the same as steps S1015 and S1017 in the flowchart of FIG. 11, so description thereof will be omitted.

According to the above-described second embodiment, the following action and effect can be obtained in addition to the action and effect obtained by the first embodiment.
(1) The image generation device 2 of the present embodiment uses the plurality of extracted pixels Pxd based on the candidate pixel parameter d corresponding to the selected pixel Pxd when the plurality of selected pixels Pxd are set from the plurality of candidate pixels Pxc. A reliability calculation unit 515 for calculating the reliability as to whether or not corresponds to the extraction target. As a result, information on whether or not the extracted pixel row of the extracted pixel Pxe has been accurately extracted can be obtained, and processing using the information can be performed.

(2) The image generation apparatus 2 of the present embodiment includes the determination unit 516 that determines whether or not the plurality of extracted pixels Pxe correspond to the continuous extraction target based on the reliability. Thereby, the extraction pixel Pxe having high reliability can be used based on the reliability determination, and the extraction accuracy can be improved.

(3) In the image generation device 2 of the present embodiment, the determination unit 516 sets the first end condition when the candidate pixel Pxc designated by the candidate pixel designation unit 601 corresponds to at least one of the end point pixels Pxt. When the pixel setting process is completed, reliability determination is performed. As a result, the calculation amount can be reduced by not performing the reliability determination when the end pixel Pxt and the connected pixel region Dc are not connected as in the case where the setting of the set pixel Pxd is ended by the second end condition. You can

(4) In the image generation device 2 of the present embodiment, the pixel extraction unit 600 performs the selected pixel setting process with at least a part of the connected pixel region Dc as the end point pixel Pxt. Accordingly, even when the extraction target such as the neurite Nr has a branch, it is possible to accurately set the connected pixel region Dc corresponding to the extraction target by extracting the extraction pixel Pxe a plurality of times.

(5) The image generation device 2 of the present embodiment includes the candidate/determined area setting unit 517 that generates data corresponding to the confirmed area image Gd indicating the confirmed area Fd confirmed as the portion corresponding to the extraction target. As a result, the fixed area Fd can be shown to the user in an easy-to-understand manner and can be used for image processing.

(6) In the image generation apparatus 2 of the present embodiment, the confirmed area image Gd is a binarized image, and the candidate/determined area setting unit 517 is binarized based on the confirmed connected pixel area Dc. Update the image. As a result, the amount of information and the amount of calculation can be reduced and the processing can be performed efficiently.

The following modifications are also within the scope of the present invention, and can be combined with the above-described embodiment.
(Modification 1)
In the above-described embodiment, the case where one cell body So is shown in the probability distribution image Gp has been described. Even when a plurality of cell bodies So are shown in the probability distribution image Gp, it is possible to similarly set the connected pixel region Dc with these cell bodies So as the end point pixels Pxt. However, in this case, the starting point pixel setting unit 514 preferably sets the starting point pixel Pxi at a position based on the center of gravity of the candidate region Fc.

FIG. 19 is a conceptual diagram for explaining the setting of the starting point pixel Pxi of this modification. In the search region D1, the pixel regions corresponding to the cell bodies So1 and So2 and the candidate regions Fc5, Fc6, Fc7 and Fc8 which are not connected to each other are shown. When the plurality of end point pixels Pxt connected to each other are called end point pixel areas, each of the cell bodies So1 and So2 corresponds to two end point pixel areas that are not connected to each other.

Here, in the pixel row of the extracted pixels Pxe (hereinafter referred to as the extracted pixel row) having the start points in the same candidate region Fc, the value of the reliability parameter R may change depending on the position of the start point pixel Pxi. For example, consider a case where a plurality of selected pixels Pxd are set from each of the start point pixel Pxi1 and the start point pixel Pxi2 at both ends of the candidate area Fc7, and an extracted pixel row connected to the cell body So2 is obtained. In this case, the extracted pixel row extending from the starting point pixel Pxi2 to the cell body So2 passes through the region having a higher brightness value by the candidate region Fc7, and thus the reliability parameter R is the average value d _{ave of the} candidate pixel parameters d. When represented by, the value of d _ave is small, and the reliability is higher than in the case of the selection pixel setting process in which the starting point pixel Pxi1 is the starting point. Therefore, the reliability of the extracted pixel Pxe to be extracted varies depending on the position of the starting point pixel Pxi set in the candidate region Fc, and the candidate region Fc can be connected to a different cell body So.

In order to suppress such variation in reliability, when there are a plurality of end point pixels Pxt or end point pixel areas that are not connected to each other in the search area D1, the start point pixel setting unit 514 sets the start point pixel Pxi to the center of gravity of the candidate area Fc. Set the position based on. For example, the starting point pixel setting unit 514 calculates the center of gravity of the candidate area Fc and sets the pixel including the center of gravity as the starting point pixel Pxi. When there is no pixel including the center of gravity, the starting point pixel setting unit 514 can set the pixel closest to the center of gravity as the starting point pixel Pxi.

In the image generation device of this modification, the end point pixel Pxt includes a plurality of pixels or pixel regions that are not connected to each other, and the start point pixel setting unit 514 has the same value in the binarized image indicating the candidate region Fc. The starting point pixel Pxi is set based on the center of gravity of the candidate region Fc including a plurality of connected pixels. As a result, it is possible to increase the possibility that the connected pixel region Dc is connected to the appropriate end pixel Pxt.

(Modification 2)
In the above-described embodiment, the end point pixel setting unit 513 is configured to extract the pixel corresponding to the cell body So from the probability distribution image Gp by image processing and set it as the end point pixel Pxt. The end point pixel Pxt may be set using the data obtained by extracting the corresponding pixels at predetermined time intervals. In culturing the cell Ce, the cell Ce may move. Therefore, to identify the same cell Ce at a certain time (first time) and a second time different from the first time, it is possible to analyze changes in characteristics such as the morphology of the same cell Ce in a time series. It is necessary to do. The above-mentioned predetermined time interval is appropriately set to several minutes to several days or the like according to the characteristics of the cell Ce. The predetermined time interval may be changed as appropriate.

FIG. 20 is a conceptual diagram showing the data processing unit 51b according to this modification. In the present modification, the data processing unit 51b includes a tracking unit 518 that tracks the position of the cell Ce. The probability distribution image generation unit 511 acquires the captured images of the cells Ce captured at predetermined time intervals from the imaging unit 20, and generates probability distribution image data from the captured images. The tracking unit 518 extracts a pixel corresponding to the cell body So in the probability distribution image Gp, as in the case of the end point pixel setting unit 513 described above. When the movement of the cell Ce is slower than the predetermined time interval, the tracking unit 518 may extract the pixel using the extraction result performed immediately before. The end point pixel setting unit 513 can set the end point pixel Pxt based on the data obtained by the extraction of the pixels corresponding to the cell body So by the tracking unit 518.

In the image generation device of this modification, the probability distribution image generation unit 511 acquires the probability distribution image data a plurality of times at different times, and the tracking unit 518 detects the portion corresponding to the end point pixel Pxt in the probability distribution image Gp. To track. As a result, the end point pixel Pxt can be set more accurately based on the past position of the cell Ce and the like.
It should be noted that the present modification is preferably used when there are a plurality of cell bodies So, but the same operational effect can be obtained even when there is one cell body So.

(Modification 3)
In the above-described embodiment, the data processing unit may include a crossover determination unit that determines the presence of a crossover of the linear portion to be extracted such as the neurite Nr.

FIG. 21 is a conceptual diagram showing a data processing unit 51c according to this modification. In this modification, the data processing unit 51c includes a crossover determination unit 519 that determines whether or not a crossover exists. The crossover determination unit 519 functions as a crossover detection unit that detects a crossover.

FIG. 22A is a conceptual diagram showing an example of the candidate region Fc when the neurites intersect. The neurite Nr1 extending from the cell body So1 corresponds to the candidate regions Fc21 and Fc23 and part of the candidate region Fc22. The neurite Nr2 extending from the cell body So2 corresponds to a part of the candidate region Fc22 and the candidate region Fc24. The intersection J1 exists in the candidate area Fc22.

When the search area D1 includes a plurality of unconnected end point pixels or end point pixel areas, the pixel extraction unit 600 first sets a plurality of selected pixels Pxd starting from the start point pixel Pxi and sets a plurality of selected pixels Pxd as in the above embodiment. Extraction pixels are extracted.

FIG. 22B is a conceptual diagram showing the connected pixel area Dc21 set by the area setting unit 512 based on the extracted extraction pixel Pxe. In the example of FIG. 22B, a case is shown in which the connected pixel region Dc21 that connects the starting point pixel Pxi and the cell body So1 is set.

When the connected pixel region Dc21 is set, the pixel extraction unit 600 stores the extracted extracted pixel Pxe in the storage unit 43 and then sets the selected pixel Pxd and the extracted pixel Pxe to the unset state. After that, the end point pixel setting unit 513 removes the cell body So1 to which the connected pixel region Dc21 is connected from the end point pixel Pxt, and extracts the extracted pixel Pxe with the cell body So2 as the end point pixel Pxt.

FIG. 22C is a conceptual diagram showing the connected pixel area Dc22 set by the area setting unit 512 based on the extracted pixel Pxe extracted when the cell body So1 is excluded from the end point pixel Pxt. In the example of FIG. 22C, the connected pixel area Dc22 that connects the starting point pixel Pxi and the cell body So2 is set.

The crossover determination unit 519 detects a pixel or a pixel region corresponding to the crossover J1 based on the set connected pixel area Dc21 and connected pixel area Dc22. The intersection determination unit 519 is a pixel corresponding to an intersection of a pixel at the end opposite to the start point pixel Pxi or a pixel region in a predetermined range from the end in the overlapping region of the connected pixel region Dc21 and the connected pixel region Dc22. And The crossover determination unit 519 may generate data indicating a point where the candidate region Fc22 has a crossover. The predetermined range may be appropriately determined based on the width of the connected pixel area Dc21 or 22 or the like.

Next, the point of detecting a pixel corresponding to the intersection of the linear portions to be extracted when the end point pixel Pxt consists of one pixel or a connected pixel area will be described. The following method is suitable when the end point pixel Pxt is one pixel or a connected pixel area, but is also applicable when the end point pixel Pxt is composed of a plurality of unconnected pixels or pixel areas.

FIG. 23(A) is a conceptual diagram showing a candidate region Fc when two neurites Nr3 and Nr4 extending from one cell body So intersect. The neurite Nr3 extending from the cell body So corresponds to the candidate regions Fc31, Fc33, and Fc34 and part of the candidate region Fc32. The neurite Nr4 extending from the cell body So corresponds to a part of the candidate region Fc32 and the candidate regions Fc35 and Fc36. The intersection J2 exists in the candidate area Fc32.

When the search area D1 includes one end point pixel Pxt or a connected end point pixel area, the pixel extracting unit 600 first sets and extracts a plurality of selected pixels Pxd starting from the start point pixel Pxi as in the above-described embodiment. Extract Pxe.

FIG. 23B is a conceptual diagram showing the connected pixel area Dc31 set by the area setting unit 512 based on the extracted extraction pixel Pxe. In the example of FIG. 23B, a case is shown in which a connected pixel region Dc31 that connects the starting point pixel Pxi and the cell body So that passes through the candidate regions Fc31, Fc32, Fc33, and Fc34 is set.

When the connected pixel area Dc31 is set, the selected pixel setting unit 602 stores the extracted extracted pixel Pxe in the storage unit 43, and then sets the selected pixel Pxd and the extracted pixel Pxe to the unset state. After that, the region setting unit 512 removes a part of the candidate region Fc corresponding to the connected pixel region Dc31 from the search region D1. A part of the candidate area Fc excluded from the search area D1 will be referred to as an exclusion area hereinafter. For example, the region setting unit 512, in the already set connected pixel region Dc31, the candidate region included in the region included in the connected element pixel region De (FIG. 10) corresponding to the predetermined number of extracted pixels Pxe on the end point pixel Pxt side. Fc is excluded from the search area D1 as an exclusion area.

FIG. 23C is a conceptual diagram schematically showing the exclusion area Dex. The pixel extraction unit 600 excludes the candidate area Fc in the portion surrounded by the dotted line from the search area D1 as the exclusion area Dex, and sets the selected pixel Pxd with the start point pixel Pxi as the start point.

FIG. 23D is a conceptual diagram showing the connected pixel area Dc32 set by the area setting unit 512 based on the extracted pixel Pxe extracted when the excluded area Dex is excluded from the search area D1. In the example of FIG. 23(D), a connected pixel region Dc32 that connects the starting point pixel Pxi and the cell body So through the candidate regions Fc31, Fc32, and Fc35 is set.

The intersection determination unit 519 detects a pixel or a pixel area corresponding to the intersection J2 based on the set connected pixel area Dc31 and connected pixel area Dc32. The crossing determination unit 519 is a pixel corresponding to a pixel at the end opposite to the start point pixel Pxi or a pixel region within a predetermined range from the end in the overlapping region of the connected pixel region Dc31 and the connected pixel region Dc32. And The crossing detection unit 519 may generate data indicating a crossing point in the candidate region Fc32. The above-mentioned predetermined range may be appropriately determined based on the width of the connected pixel region Dc31 or 32 or the like.

If there is no crossover, the selected pixel setting process ends due to the second end condition, and the extracted pixel string as shown in FIG. 22C or 23D cannot be obtained. As described above, the crossing determination unit 519 determines the presence/absence of a crossover, and when a crossover is detected, a pixel corresponding to the crossover is derived, and thus, morphological analysis and the like can be performed using these pieces of information. .. Further, the image generation unit 700 may indicate the position of intersection and the like in the connected region image Gs.

In the image generation apparatus of this modification, the end point pixel Pxt includes a plurality of pixels or pixel regions that are not connected to each other, and the crossover determination unit 519 determines whether or not the linear portion to be extracted crosses and detects the crossover. .. Accordingly, it is possible to provide information about the position of the intersection in the input image in which the portions (cell bodies So1, So2, etc.) corresponding to the plurality of unconnected end pixel regions and the like are shown. As a result, the pixel corresponding to the neurite Nr can be accurately extracted even in the situation where there is a crossover. The positional information including the information about the crossover enables more accurate quantification of the cell-cell interaction.

In the image generation device of the present modification, the pixel extraction unit 600 excludes a region including some of the extracted pixels Pxe set in the selection pixel setting process, performs the selection pixel setting process again, and the crossing detection unit 519. Detects the intersection of the linear parts to be extracted. Thereby, even when only one end point pixel Pxt or a connected end point pixel area is present in the input image, it is possible to provide information about the position of the intersection in the input image.

(Modification 4)
In the above-described embodiment, the data processing unit may include a width detection unit that detects the width of the linear portion to be extracted such as the neurite Nr.

FIG. 24 is a conceptual diagram showing the data processing unit 51d according to this modification. In the present modification, the data processing unit 51d includes a width detection unit 520 that detects the width of the linear portion to be extracted.

FIG. 25 is a conceptual diagram for explaining the point of detecting the width of the extraction target linear portion (neurite Nr) based on the connected pixel region Dc. It is assumed that the user inputs via the input unit 41 to calculate the width of the neurite Nr at the position of the point T (hereinafter referred to as the input point T) in the connected pixel region Dc. In this case, the width detection unit 520 calculates the circle Ci having the largest radius r among the circles inscribed in the connected pixel region Dc including the input point T, and sets twice the radius r as the width Wd of the neurite Nr. calculate. Information about the calculated width Wd is output from the output unit 44.

In the image generation apparatus of this modification, the width calculation unit 520 calculates the width Wd of the extraction target linear portion. As a result, it is possible to provide information regarding the shape such as the thickness of the linear portion to be extracted.

In the image generation device of the present modification, the width calculation unit 520 sets the width Wd of the connected pixel region Dc corresponding to the input point T in the connected pixel region Dc to the maximum included in the connected pixel region Dc including the input point T. It is calculated by the diameter of the circle Ci. Accordingly, the width Wd of the extraction target linear portion can be calculated.

(Modification 5)
In the modification described above, the width of the neurite Nr is calculated using the inscribed circle of the connected pixel region Dc, but the width Wd of the neurite Nr is calculated based on the length of the line segment passing through the connected pixel region Dc. May be.

FIG. 26 is a conceptual diagram for explaining that the width of the linear portion (neurite Nr) to be extracted is detected based on the connected pixel area Dc. It is assumed that the user inputs via the input unit 41 so as to calculate the width of the neurite Nr at the position of the input point T. In this case, the width detection unit 520 assumes line segments L1, L2, and L3 that extend through the input point T in a plurality of different directions, and the line segments L1, L2, and L3 have a length included in the connected pixel region Dc. To calculate. The width detection unit 520 sets the shortest value among the calculated lengths as the width Wd of the neurite Nr. Note that the directions of the line segments L1, L2, and L3 are not particularly limited, and can be set as appropriate, such as shifting by a certain angle. Further, the number of line segments used for calculating the width Wd is not particularly limited.

In the image generation device of the present modification, the width calculation unit 520 determines the width Wd of the connected pixel area corresponding to the input point T in the connected pixel area Dc at the plurality of line segments L1, L2, and L3 passing through the input point T. It is calculated based on the shortest length included in the connected pixel region Dc. Thereby, the width Wd of the extraction target linear portion can be calculated while suppressing the calculation amount.

(Modification 6)
In the above-described embodiment, when outputting the brightness value at the point input by the user in the input image or the probability distribution image Gp, the brightness value may be calculated based on the brightness of a plurality of pixels and output. This makes it possible to suppress variations and provide more accurate brightness.

FIG. 27 is a conceptual diagram for explaining a method of calculating an output brightness value (hereinafter, referred to as an output brightness value). When the output brightness value is calculated for the input point T designated by the user via the input unit 41, the data processing unit 51a sets a predetermined range based on the position of the input point T (hereinafter, referred to as a corresponding pixel area Ct). An average such as an arithmetic average of the brightness values of the pixels is calculated as the output brightness value. The calculated output brightness value is output from the output unit 44. The shape and size of the corresponding pixel area Ct are not particularly limited, and may be a pixel area or the like corresponding to the range of a circle having a radius of several pixels centered on the input point T.

-Third Embodiment-
The image generation device 3 of the third embodiment has the same configuration as the image generation device 2 according to the second embodiment, but the configuration of the data processing unit is different from that of the second embodiment. In the third embodiment, a method will be described in which the selected pixel setting processing for a plurality of start point pixels Pxi is performed in parallel without sacrificing accuracy. The same parts as those of the second embodiment are referred to by the same reference numerals as those of the second embodiment, and the description thereof will be omitted depending on the case.

FIG. 28 is a conceptual diagram showing the configuration of the image generating apparatus 3 of this embodiment. The image generation device 3 includes the components other than the control unit 50a and the data processing unit 51e in the same manner as in FIG. 1, but the illustration is omitted. The image generating apparatus 3 is different from the image generating apparatus 2 of the above-described embodiment in that the area setting unit 512a includes a starting point arrangement area setting unit 531 and a candidate pixel arrangement area setting unit 532. Further, the control unit 50a of the image generating apparatus 3 includes a CPU suitable for parallel processing such as a multi-core CPU, and the selected pixel setting process performed by the data processing unit 51e is performed by the CPU suitable for this parallel processing. It is preferred that

In the following, a method of setting the connected pixel area Dc when the probability distribution image Gp includes a portion corresponding to a plurality of cell bodies So will be described. The image generation method according to the present embodiment is suitable for a case where a plurality of cell bodies So exist or the neurite Nr extends in a complicated manner, but it is not particularly limited to such a case and can be applied. Is.

In the above embodiment, the starting point pixel Pxi and the candidate pixel Pxc are set in the search area D1 set by the area setting unit 512. In the present embodiment, the starting point pixel Pxi is set in the starting point arrangement area set by the starting point arrangement area setting unit 531 and the candidate pixel Pxc is set in the candidate pixel arrangement area set by the candidate pixel arrangement area setting unit 532.

FIG. 29 is a conceptual diagram showing the starting point arrangement area. The start point arrangement area 80 includes a first start point arrangement area 81 and a second start point arrangement area 82. The first start point arrangement area 81 includes a first connection area 81a, a second connection area 81b, a third connection area 81c, and a fourth connection area 81d that are not connected to each other but are one continuous area. .. The second starting point arrangement area 82 includes a first connecting area 82a, a second connecting area 82b, a third connecting area 82c, and a fourth connecting area 82d that are not connected to each other but are one continuous area. ..

Below, the 1st connection field 81a, the 2nd connection field 81b, the 3rd connection field 81c, and the 4th connection field 81d, and the 2nd connection field 82a, the 2nd connection field 82b, the 3rd connection field 82c, and the 4th connection field. When referring to the individual regions without distinguishing each of the regions 82d, they are simply referred to as a connecting region.

Reference numerals

81a, 81b, 81c and 81d are described as 81a to d, and

reference numerals

82a, 82b, 82c and 82d are described as 82a to d.

As will be described later, a selection pixel setting process starting from the start point pixels Pxi1a, Pxi1b, Pxi1c and Pxi1d (hereinafter referred to as Pxi1a to d) arranged in the first start point arrangement region 81 and the second start point arrangement region 82. The selection pixel setting process starting from the start point pixels Pxi2a, Pxi2b, Pxi2c, and Pxi2d (FIG. 30) arranged in 1 is not performed in parallel at the same time. In this way, the start point arrangement area 80 has a plurality of areas (the first start point arrangement area 81 and the second start point 81) based on when the selection pixel setting processing corresponding to the start point pixels Pxi arranged in the start point arrangement area 80 is performed. It is divided into placement areas 82).

A predetermined minimum connection area interval between the connection areas 81a to 81d included in the first start point arrangement area 81 and between the connection areas 82a to 82d included in the second start point arrangement area 82 is set to M. Then, they are separated by at least M pixels or more. In the example of the starting point arrangement area 80 in FIG. 29, the number of pixels in the vertical direction of each of the connection areas 81a to 81d is equal to the number of pixels in the vertical direction of the probability distribution image Gp, and the number of pixels in the horizontal direction is Mi. ing. Mi has a value equal to or larger than M.

At least a part of the selected pixel setting process starting from the starting point pixels Pxi1a to Pxi1 respectively arranged in the connection regions 81a to 81d is performed in parallel at the same time. Therefore, by separating at least M pixels between the connection regions 81a to 81d, the pixel row of the selected pixel Pxd starting from different start point pixels Pxi passes through the same candidate region Fc (FIG. 17) of the probability distribution image Gp. It is possible to reduce the possibility that The same applies to the connection regions 82a to 82d. As a result, the candidate area Fc and the finalized area Fd (FIG. 17) are appropriately updated, and even if the portion corresponding to the extraction target has a complicated structure, the connected pixel area Dc can be set quickly without lowering the accuracy. It can be carried out.

The minimum connected region interval M is, for example, the effective magnification at the time of imaging by the imaging unit 20, the pixel width of the probability distribution image Gp, and the statistical value of the length of the linear portion obtained in the past for the subject of the input image. It is calculated based on For example, if the subject is a nerve cell of a specific type, the upper limit of the length of the neurite Nr in the nerve cell of the type is Lmax, the effective magnification is x, and the width of the pixel is y, M=2×Lmax× The connected region interval M can be calculated by the formula of x/y. As an example, when Lmax=500 μm, x=10 (times), and y=8 μm, the minimum connected region interval M is 1250 pixels. As a result, the neurite Nr extends straight from the cell body So in the connecting region (for example, the connecting region 82b) between the starting point pixels Pxi in the two connecting regions (for example, the connecting

regions

81a and 81b). Even in such a case, it is possible to reduce the possibility that the pixel rows of the selected pixels Pxd set in parallel at the same time pass through the same candidate pixel Pxc.
When the minimum connected region interval M is set based on parameters such as the pixel width of the probability distribution image Gp, the minimum connected region interval M is set based on the width of each cell Ce and the length of the neurite Nr. Good. Further, the minimum connection area interval M can be determined based on the connection areas 81a to 81d and 82a to 82d from the viewpoint of efficiency.

The starting point arrangement area setting unit 531 sets the starting point arrangement area 80, the first starting point arrangement area 81, the second starting point arrangement area 82, and the connection areas 81a to d, 82a to d based on the input from the input unit 41 and the like. .. Image data indicating image portions corresponding to these regions in the probability distribution image Gp, or information regarding the shape or size of the image portion is input from the input unit 41. Based on this image data and information, the starting point arrangement area setting unit 531 determines that each pixel in the probability distribution image Gp has a starting point arrangement area 80, a first starting point arrangement area 81, a second starting point arrangement area 82, and connecting areas 81a to 81d. 82a to 82d are set to be included or not included.

The candidate pixel arrangement area setting unit 532 sets the candidate pixel arrangement area 90, which is an area in which the candidate pixel Pxc is arranged in the selection pixel setting process. The candidate pixel arrangement area 90 is set based on an input from the input unit 41 or the like. In FIG. 29, the candidate pixel arrangement area 90 is set for the entire probability distribution image Gp. The area where the candidate pixel Pxc is set is an area where the selection pixel Pxd is set and the extraction pixel Pxe is extracted.

The starting point pixel setting unit 514 sets the starting point pixel Pxi in the starting point arrangement area 80. The starting-point pixel setting unit 514 arranges the starting-point pixels Pxi1a-d in the first starting-point arrangement region 81 when performing the selection pixel setting process with the first starting-point arrangement region 81 as the starting point. The starting-point pixel setting unit 514 arranges the starting-point pixels Pxi2a to Pxi2d in the second starting-point arrangement region 82 when performing the selection pixel setting process with the second starting-point arrangement region 82 as the starting point.

The pixel extraction unit 600 performs a plurality of selected pixel setting processes simultaneously in parallel based on the position of the starting point pixel Pxi. The area setting unit 512a sets the connected pixel area Dc based on the extracted pixel Pxe extracted from the selected pixels Pxd set by the plurality of selected pixel setting processes.

In FIG. 29, the pixel extraction unit 600 simultaneously performs a plurality of selected pixel setting processes (hereinafter, referred to as first selected pixel setting process) starting from the starting point pixels Pxi1a to Pd1 arranged in the first starting point arrangement area 81. The points to be performed are schematically shown. In FIG. 29, a connected pixel area Dc1a connecting the starting point pixel Pxi1a and the cell body So1a, a connected pixel area Dc1b connecting the starting point pixel Pxi1b and the cell body So1b, and a connected pixel connecting the start point pixel Pxi1c and the cell body So1c. A region Dc1c and a connected pixel region Dc1d that connects the starting point pixel Pxi1d and the cell body So1d are shown.

The pixel extraction unit 600 performs a plurality of selection pixel setting processes (hereinafter, second selection pixel setting processes) starting from the start point pixels Pxi2a to Pxi2a to d arranged in the second start point arrangement region 82 at different times from the first selection pixel setting process. (Referred to as setting processing) is performed in parallel at the same time.

FIG. 30 is a conceptual diagram schematically showing that the pixel extraction unit 600 simultaneously performs a plurality of second selected pixel setting processes in parallel. In FIG. 30, a connected pixel area Dc2a connecting the starting point pixel Pxi2a and the cell body So2a, a connected pixel area Dc2b connecting the starting point pixel Pxi2b and the cell body So2b, and a connected pixel connecting the start point pixel Pxi2c and the cell body So2c. A region Dc2c and a connected pixel region Dc2d that connects the starting point pixel Pxi2d and the cell body So2d are shown.

The area setting unit 512a integrates a plurality of connected pixel areas Dc (FIG. 4) obtained based on a plurality of selected pixel setting processings performed in parallel at the same time. In this integration, one connected connected pixel area Dc including pixels included in at least one connected pixel area Dc is newly set. The candidate/fixed area setting unit updates the candidate area Fc and the fixed area Fd (FIG. 17) based on the integrated connected pixel area Dc.

FIG. 31 is a flowchart showing the flow of the image generation method according to this embodiment. Since steps S4001 to S4007 are the same as steps S1001 to S1007 in the flowchart of FIG. 11, description thereof will be omitted. When step S4007 ends, step S4009 starts.

In step S4009, the starting point placement area setting unit 531 sets the first starting point placement area 81, the second starting point placement area 82, and the connection areas 81a-d, 82a-d. When step S4009 ends, step S4011 starts. In step S4011, the starting point pixel setting unit 514 sets the starting point pixel Pxi in the starting point arrangement region 80, the pixel extraction unit 600 sets a plurality of selected pixels Pxd to extract a plurality of extracted pixels Pxe, and the region setting unit 512a The parallel processing for setting the connected pixel area Dc is performed, and the area setting unit 512a sets the integrated connected pixel area. When step S4011 ends, step S4013 starts.

In step S4013, the image generation unit 700 generates connected area image data corresponding to the connected area image Gs indicating the integrated connected pixel area. When step S4013 ends, step S4015 starts. In step S4015, the output unit 44 outputs the connected region image Gs. When step S4015 ends, the process ends.

FIG. 32 is a flow chart showing the flow of step S4011 in the flow chart of FIG. In the flowchart of FIG. 32, the selected pixel setting process for the first connection area 81a and the second connection area 81b of the first start point arrangement area 81 and the first connection area 82a and the second connection area 82b of the second start point arrangement area 82 are performed. The point that the selected pixel setting process regarding the above is performed concurrently in parallel with each other, and the description about the selected pixel setting process regarding other connected regions is omitted. Upon completion of step S4009, steps S501a and S501b are started.

In step S501a, the starting point pixel setting unit 514 sets the starting point pixel Pxi in the first connection area 81a of the first starting point arrangement area 81. When step S501a ends, step S503a starts. In step S503a, the pixel extraction unit 600 sets a plurality of selected pixels Pxd starting from the start point pixel Pxi set in step S501a and extracts the extracted pixel Pxe, and the area setting unit 512a sets the extracted pixel Pxe to Based on this, the connected pixel area Dc is set.
In the setting of the candidate pixel Pxc of the present embodiment, the value of the candidate pixel position parameter N is not particularly limited and can be set to any value of 1 or more. The same applies to the following selected pixel setting processing.

In step S501b, the starting point pixel setting unit 514 sets the starting point pixel Pxi in the second connection area 82a of the second starting point arrangement area 82. In step S503b, the pixel extraction unit 600 sets a plurality of set pixels Pxd starting from the start point pixel Pxi set in step S503a, extracts the extracted pixel Pxe, and the area setting unit 512a sets the extracted pixel Pxe to the extracted pixel Pxe. Based on this, the connected pixel area Dc is set. When step S503a and step S503b are completed, step S505 is started.

In step S505, the area setting unit 512a integrates the connected pixel areas Dc, and the candidate/fixed area setting unit 517 updates the candidate area Fc and the fixed area Fd. When step S505 ends, steps S507a and S507b start.

In step S507a, the starting point pixel setting unit 514 sets the starting point pixel Pxi in the first connection area 82a of the second starting point arrangement area 82. When step S507a ends, step S509a starts. In step S509a, the pixel extraction unit 600 selects a plurality of selected pixels Pxd starting from the start point pixel Pxi set in step S507a, extracts the extracted pixel Pxe, and the area setting unit 512a selects the extracted pixel Pxe. Based on this, the connected pixel area Dc is set.

In step S507b, the starting point pixel setting unit 514 sets the starting point pixel Pxi in the second connection area 82b of the second starting point arrangement area 82. When step S507b ends, step S509b starts. In step S509b, the pixel extraction unit 600 sets a plurality of selected pixels Pxd starting from the start point pixel Pxi set in step S507b and extracts the extracted pixel Pxe, and the area setting unit 512a sets the extracted pixel Pxe to the extracted pixel Pxe. Based on this, the connected pixel area Dc is set. When step S509a and step S509b are completed, step S511 is started.

In step S511, the area setting unit 512a integrates the connected pixel areas Dc and sets the integrated connected pixel area. When step S511 ends, step S4013 starts.

According to the above-described third embodiment, the following action and effect can be obtained in addition to the action and effect obtained by the first and second embodiments.
(1) In the image generation device 3 of the present embodiment, the pixel extraction unit 600 performs the selected pixel setting process on the plurality of start point pixels Pxi1a to Pxi1d arranged in the connection regions 81a to 81d which are not connected to each other, by performing the above connection This is performed at a different time from the selected pixel setting process for the start point pixels Pxi2a to Pxi2a to d arranged in the connection regions 82a to 82d that are the start point arrangement region 80 excluding the regions 81a to 81d. Accordingly, even if the extraction target has a complicated structure, it is possible to quickly extract the pixel corresponding to the extraction target without lowering the accuracy.

(2) In the image generation device 3 of the present embodiment, the interval between the connected regions 81a to 81d is set based on the length of the linear portion to be extracted. This makes it possible to set an appropriate interval Mi between the connected regions 81a to 81d in accordance with the characteristics of the linear portion to be extracted, and it is possible to more quickly extract the pixel corresponding to the extraction target.

The following modifications are also within the scope of the present invention, and can be combined with the above-described embodiment.
(Modification 1)
In the above-described embodiment, the connection regions 81a to 81d and 82a to d are set like the first start point arrangement region 81 and the second start point arrangement region 82 in FIG. 30, but the shape and size of the connection region is different. There is no particular limitation as long as they are separated from each other by at least the minimum connection region interval M.

FIG. 33 is a conceptual diagram showing an example of a connected area. In the probability distribution image Gp, the third start point arrangement area 83 and the fourth start point arrangement area 84 corresponding to the selected pixel setting processing at different times are set as the start point arrangement area 80. The third starting point arrangement area 83 includes

rectangular connection areas

83a, 83b, 83c, and 83d arranged at the four corners of the probability distribution image Gp, and a cross-shaped connection area 83e that extends point-symmetrically from the center of the probability distribution image Gp. With. The fourth starting point arrangement area 84 is an L-shaped or rotated L-shaped

connection area

84a, 84b, 84c located between each of the

connection areas

83a, 83b, 83c and 83d and the connection area 83e. 84d.

The spacing Mi2 between each of the

coupling regions

83a, 83b, 83c, and 83d and the coupling region 83e, and the spacing Mi1 between the

coupling regions

84a, 84b, 84c, and 84d that are vertically and horizontally adjacent to each other are the extraction target linear shape. Based on the length of the portion, it is set to be larger than the minimum connected region interval M.
It should be noted that the density of the cells Ce is detected by a known method such as the density of the cell bodies So, and the selected pixel setting processing is performed in parallel at the same time so that the dense areas fit into one connected area. You may set the space|interval of the some connection area performed. Thereby, parallel processing can be efficiently performed.

(Modification 2)
In the above-described embodiment, the pixel extraction unit 600 extracts the extracted pixel Pxe based on the Dijkstra method, but the extracted pixel Pxe may be extracted by the Astar algorithm. The Astar algorithm differs from the above-described method in that when the selection pixel Pxd is set, the selection pixel Pxd is set using the distance between the candidate pixel Pxc and the end point pixel Pxt.

(Modification 3)
In the above-described embodiment, the pixel extraction unit 600 extracts the extracted pixels Pxe based on the Dijkstra method. Pxe may be extracted.

(Modification 4)
In the above-described embodiment, the pixel extraction unit 600 extracts the extracted pixel Pxd based on the Dijkstra method. However, a predetermined parameter (hereinafter referred to as energy) is defined for each pixel combination, and a combination of a plurality of pixels that maximizes or minimizes energy is calculated as the extracted pixel Pxe under a predetermined condition. Good.

For each pixel of the probability distribution image Gp, a value of 1 is extracted when the pixel is extracted as the extracted pixel Pxe, and a value of 0 is associated when it is not extracted as the extracted pixel Pxe. In this case, assuming that the probability distribution image Gp is composed of 100 pixels, the extraction of the pixel row of the plurality of extraction pixels Pxe corresponds to searching for an appropriate combination from 2 100 combinations.

Regarding whether to be extracted as the extracted pixel Pxe, the correlation between a plurality of pixels is not considered. In this case, the energy of the i-th pixel in the probability distribution image Gp is set to ei, and a combination such that the sum of the energy ei of the extracted pixel Pxe becomes an optimized value such as a maximum value or a minimum value may be calculated. .. For example, when the energy is the brightness value of the pixel, the combination of pixels in which the sum of the energy has the maximum value may be calculated. When the reciprocal of the brightness value is used as energy, or when the image is inverted in black and white, the combination with the minimum energy sum may be calculated. In such optimization, a condition indicating a constraint based on the connectivity between the start point and the end point may be added as appropriate. The same applies when considering the correlation between a plurality of pixels below.

The correlation between multiple pixels shall be taken into consideration regarding whether or not to be extracted as the extracted pixel Pxe. In this case, the energy for both the i-th pixel and the j-th pixel in the probability distribution image Gp being the extracted pixel Pxe is eij, and the energy eij of any pair of pixels included in the combination of the extracted pixels Pxe is It suffices to calculate a combination in which the sum is an optimized value such as a maximum value or a minimum value.

(Modification 5)
In the above-described embodiment, the first selection pixel setting process and the second selection pixel setting process are alternately performed. However, the first selection pixel setting process or the second selection pixel setting process may be continuously performed, or any one of them may be configured to be performed based on a predetermined condition. In the present modification, either the first start point arrangement area 81 or the second start point arrangement area 82 is selected based on the length of the neurite corresponding to each candidate area Fc, and the selected pixel setting process in the selected area is performed. Then, the extraction pixel Pxe is extracted. The flow of the image generation method according to this modification is the same as the flow shown in the flowchart of FIG. 31 of the third embodiment, but the part corresponding to step S4011 is different.

FIG. 34 is a flowchart showing the flow of the part corresponding to step S4011 in the flowchart of FIG. 31, in the present modification. When step S4009 (FIG. 31) is completed, step S601 is started. In step S601, the data processing unit 51e acquires, for each candidate region Fc, information about the length of the corresponding neurite Nr. The data processing unit 51e uses the length of the neurite Nr corresponding to each pixel region fragment F (FIG. 3) obtained by analyzing the probability distribution image Gp as the length of the neurite Nr corresponding to the candidate region Fc. get. The method of calculating the length of the neurite Nr corresponding to the candidate region Fc is not particularly limited. When step S601 ends, step S602 starts.

In step S602, the pixel extraction unit 600, if the starting point pixel Pxi is set, the area including the starting point pixel Pxi corresponding to the longest neurite in the first starting point arrangement area 81 and the second starting point arrangement area 82. Select. When step S602 ends, step S603 starts. In step S603, the starting point pixel setting unit 514 sets at least a part of the starting point pixels Pxi that can be set in the region selected in step S602, and the selected pixel setting unit 602 sets a plurality of selected pixels Pxd, The extraction unit 600 extracts the extracted pixel Pxe, and the region setting unit 512a sets the connected pixel region Dc. For example, the starting point pixel setting unit 514 can set the starting point pixel Pxi corresponding to the longest neurite in each connected region in the selected region based on the information acquired in step S601. When step S603 ends, step S604 starts.

Since step S604 is the same as step S505 in the flowchart of FIG. 32, description thereof will be omitted. When step S604 ends, step S605 starts. In step S605, the pixel extraction unit 600 determines whether the candidate area Fc still remains. If the candidate region Fc still remains, the pixel extraction unit 600 makes an affirmative decision in step S605 and returns to step S602. When no candidate region Fc remains, the pixel extraction unit 600 makes a negative determination in step S605 and starts step S606. Since step S606 is the same as step S511, description thereof will be omitted. When step S606 ends, the process ends.

In the image generation method of the present modification, the length of the neurite Nr corresponding to each of the start point pixels Pxi included in the first start point arrangement area 81 or the second start point arrangement area 82, or the connection areas 81a to d, 82a to d is set. Based on this, the order in which the selected pixel setting process is performed is set. Thereby, it is possible to efficiently extract pixels in a desired order based on the length of the neurite Nr obtained in advance.

(Modification 6)
A program for realizing the information processing function of the information processing apparatus 40 of the above-described embodiment is recorded in a computer-readable recording medium, and the setting of the selected pixel Pxd and the extracted pixel recorded in the recording medium are performed. A computer system may be caused to read and execute a program relating to processing by the

data processing units

51, 51a, 51b, 51c, 51d and 51e such as Pxe extraction and setting of the connected pixel region Dc. The “computer system” mentioned here includes an OS (Operating System) and hardware of peripheral devices. The "computer-readable recording medium" refers to a portable recording medium such as a flexible disk, a magneto-optical disk, an optical disk, a memory card, or a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" means to hold a program dynamically for a short time like a communication line when transmitting the program through a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside the computer system that serves as a server or a client in that case may hold a program for a certain period of time. Further, the above program may be for realizing a part of the above-described functions, and may be for realizing the above-mentioned functions in combination with a program already recorded in the computer system. ..

When applied to a personal computer (hereinafter referred to as a PC) or the like, the above-mentioned control-related program can be provided through a recording medium such as a CD-ROM or a data signal such as the Internet. FIG. 34 is a diagram showing this state. The PC 950 receives the program provided via the CD-ROM 953. Further, the PC 950 has a function of connecting to the communication line 951. The computer 952 is a server computer that provides the above program, and stores the program in a recording medium such as a hard disk. The communication line 951 is the Internet, a communication line for personal computer communication, or a dedicated communication line. The computer 952 reads the program using the hard disk and transmits the program to the PC 950 via the communication line 951. That is, the program is carried as a data signal by a carrier wave and transmitted through the communication line 951. As described above, the program can be supplied as a computer-readable computer program product in various forms such as a recording medium and a carrier wave.

As a program for realizing the above-described information processing function, a start point pixel setting process (corresponding to step S1009 in the flowchart of FIG. 11) of setting a start point pixel Pxi from a plurality of pixels forming the probability distribution image Gp, and the plurality of To cause the processing device to perform a pixel extraction process (corresponding to S1011) of designating a plurality of candidate pixels Pxc from a plurality of pixels excluding the start point pixel Pxi among the pixels and setting a selection pixel Pxd from the plurality of candidate pixels Pxc. Of the plurality of candidate pixels Pxc are included in the program of at least two pixels from the starting point pixel Pxi. With this, it is possible to realize the process of quickly extracting the pixel corresponding to the extraction target while suppressing the deterioration of accuracy.

The present invention is not limited to the contents of the above embodiment. Other modes that are conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention.

The disclosure content of the following priority basic application is incorporated herein by reference.
Japanese Patent Application No. 2019-005373 (filed on January 16, 2019)

1, 2, 3,... Image generating device, 10... Incubator, 20... Imaging unit, 40... Information processing unit, 43... Storage unit, 44... Output unit, 50, 50a... Control unit, 51, 51a, 51b, 51c , 51d, 51e... Data processing unit, 80... Start point arrangement area, 81... First start point arrangement area, 82... Second start point arrangement area, 83... Third start point arrangement area, 84... Fourth start point arrangement area, 81a, 81b , 81c, 81d, 82a, 82b, 82c, 82d, 83a, 83b, 83c, 83d, 83e, 84a, 84b, 84c, 84d... Connection region, 90... Candidate pixel arrangement region, 100... Culture part, 511... Probability distribution Image generation unit, 512, 512a... Region setting unit, 513... End point pixel setting unit, 514... Starting point pixel setting unit, 515... Reliability calculation unit, 516... Judgment unit, 517... Candidate/determined region setting unit, 518... Tracking Reference numeral 519... Crossover determination portion, 520... Width calculation portion, 600... Pixel extraction portion, 601... Candidate pixel designation portion, 602... Selected pixel setting portion, 700... Image generation portion, Ce... Cell, D1... Search area, Dc , Dc12, Dc21, Dc22, Dc3, Dc31, Dc32, Dc1a, Dc1b, Dc1c, Dc1d, Dc2a, Dc2b, Dc2c, Dc2d... Connected pixel region, Ds... Calculation pixel region, F, F1, F2, F3, F4... Pixel Region fragments, Fc, Fc1, Fc2, Fc3, Fc4, Fc5, Fc6, Fc7, Fc8, Fc21, Fc22, Fc23, Fc24, Fc31, Fc32, Fc33, Fc34, Fc35, Fc36... Candidate regions, Fd, Fd1, Fd2... Definite area, Gc... Candidate area image, Gd... Definite area image, Gp... Probability distribution image, Gs... Connected area image, J1, J2... Crossover, M... Minimum connected area interval, Nr, Nr1, Nr2, Nr3, Nr4... Neurites, Pxc, Pxc1... Candidate pixels, Pxd, Pxd0, Pxd1, Pxd2... Selected pixels, Pxe, Pxe1... Extraction pixels, Pxi, Pxi1, Pxi2, Pxi1a, Pxi1b, Pxi1c, Pxi1d, Pxi2a, Pxi2c, Pxi2b, Pxi2b, Pxi2b, Pxi2b. Start point pixel, Pxs, Pxs1, Pxs2... Calculation pixel, Pxt... End point pixel, So, So1, So2, So1a, So1b, So1c, So1d, So2a, So2b, So2c, So2d... Cell body, T... Input point, W, Wd ...Width of the neurite.

Claims

An image processing device for extracting cells by connecting fragmented cell regions in an image,
An acquisition unit that acquires a fragmented cell region from the image,
A first pixel setting unit that sets a first pixel in the fragmented cell region;
A plurality of pixel groups are set based on the position of the first pixel, a second pixel is selected based on a luminance value of pixels included in the plurality of pixel groups, and then the first pixel and the second pixel are selected. An extraction unit that connects cells to extract cells,
An image processing apparatus including.
The image processing apparatus according to claim 1,
The extraction unit is
(A) As the plurality of pixel groups, a candidate pixel group that is separated from the first pixel by a predetermined number of pixels is set, and based on the brightness value of the pixels included in the candidate pixel group, Selecting the second pixel, (b) spacing a predetermined number of pixels around the second pixel, and setting a plurality of pixels excluding the first pixel and the second pixel as a new candidate pixel group, A new second pixel is selected from the new candidate pixel group based on the brightness values of the pixels included in the new candidate pixel group, and (c) the new second pixel selection in (b) is repeatedly executed. Then, the plurality of new second pixels are sequentially selected, and then (d) the plurality of new second pixels are connected to extract a cell.
The image processing apparatus according to claim 2,
The extraction unit includes an end point pixel setting unit that sets at least one end point pixel, and sets an end condition for ending the process (c) based on the position of the end point pixel,
When the candidate pixel group set by the extraction unit or the set second pixel corresponds to at least one of the end point pixels, or the setting of the second pixel from a plurality of the candidate pixel groups is predetermined. The image processing apparatus ends the process (c), assuming that the end condition is satisfied when the process is performed a number of times.
The image processing apparatus according to claim 2 or 3,
The extraction unit is
A plurality of third pixels are set based on positions of the first pixel and the second pixel, which are excluded when the plurality of candidate pixel groups are set, and the plurality of candidate pixel groups. , A new second pixel is selected from the plurality of candidate pixel groups based on the luminance value of the third pixel, and at least a part of the plurality of second pixels obtained by the processing is extracted as an extraction pixel. Image processing device.
The image processing apparatus according to claim 4,
An image processing apparatus comprising: an area setting unit that sets at least a first pixel area including the extracted pixel in the image.
The image processing apparatus according to claim 5,
The extraction unit calculates a first value for each of the candidate pixels based on the brightness of the plurality of third pixels, and calculates a second value from the plurality of candidate pixel groups based on the first value. An image processing device for selecting pixels.
The image processing apparatus according to claim 6,
The image processing device, wherein the plurality of third pixels are pixels in a range centered on a pixel located on a line connecting the exclusion pixel and the candidate pixel group.
The image processing apparatus according to claim 6,
Based on the first value corresponding to the candidate pixel group when the plurality of second pixels are selected from the plurality of candidate pixel groups, the plurality of second pixels selected by the (c) processing Image processing apparatus for extracting at least a part of the image.
The image processing device according to any one of claims 6 to 8,
Based on the first value corresponding to the second pixel when the plurality of second pixels are set from the plurality of candidate pixel groups, the plurality of second pixels selected by the (c) processing An image processing apparatus including a reliability calculation unit that calculates a reliability as to whether at least a part of the above corresponds to an extraction target.
The image processing apparatus according to any one of claims 5 to 9,
The image is an image in which, for each pixel of the image obtained by imaging, the probability of being a pixel corresponding to the cell is calculated by using learned machine learning, and luminance is made to correspond to the probability based on the probability. Processing equipment.
The image processing apparatus according to any one of claims 5 to 10,
A storage unit that stores an area including the selected second pixels,
The image setting device, wherein the area setting unit sets a search area for extracting a plurality of new second pixels from the image after the storage unit stores the area including the plurality of second pixels.
The image processing device according to any one of claims 5 to 11,
The image processing apparatus, wherein the extraction unit performs the process (c) by using at least a part of the first pixel area as the end point pixel.
The image processing device according to any one of claims 5 to 12,
The element pixel group constituting the first pixel area is a square area having at least two pixels,
An image processing device in which at least a part of the second pixel is arranged in pixels on the outer periphery of the element pixel group.
The image processing device according to any one of claims 5 to 13,
The end point pixel includes a plurality of pixels or pixel regions that are not connected to each other,
The image processing apparatus, wherein the extraction unit includes a first intersection detection unit that detects an intersection of the linear portions to be extracted.
The image processing apparatus according to any one of claims 5 to 14,
The extraction unit stores an area including the plurality of extracted pixels set by the area setting unit in the storage unit, and then excludes the area, and again performs the process (c) to extract the linear portion to be extracted. An image processing apparatus comprising a second crossover detection unit that detects a crossover.
The image processing device according to any one of claims 5 to 15,
A width calculation unit that calculates the width of the linear portion to be extracted is provided.
The width calculation unit calculates the width of the first pixel area corresponding to a point in the first pixel area by the diameter of the largest circle included in the first pixel area including the point. ..
The image processing apparatus according to claim 16,
The width calculation unit determines the width of the first pixel region corresponding to a point in the first pixel region when the length included in the first pixel region is the shortest in a plurality of line segments passing through the point. An image processing device that calculates the length.
The image processing device according to any one of claims 5 to 17,
The extraction unit may perform the (c) processing on the plurality of first pixels arranged in a plurality of start point arrangement areas that are not connected to each other, in the first area arranged in an area excluding the plurality of start point arrangement areas. An image processing apparatus that performs processing at a different time from the processing (c) for pixels.
An image processing method for extracting cells by connecting fragmented cell regions in an image,
Region acquisition to obtain a fragmented cell region from the image,
A first pixel setting for setting a first pixel in the fragmented cell region;
A plurality of pixel groups are set based on the position of the first pixel, a second pixel is selected based on a luminance value of pixels included in the plurality of pixel groups, and then the first pixel and the second pixel are selected. Pixel extraction to connect cells and extract cells,
An image processing method including.
A program for extracting cells by connecting fragmented cell regions in an image,
An acquisition process for acquiring a fragmented cell region from the image,
A first pixel setting process for setting a first pixel in the fragmented cell region;
A plurality of pixel groups are set based on the position of the first pixel, a second pixel is selected based on a luminance value of pixels included in the plurality of pixel groups, and then the first pixel and the second pixel are selected. An extraction process that connects cells to extract cells,
A program that causes the processing device to perform.