JPWO2019176012A1 - Image processing methods, image processing devices, user interface devices, image processing systems, and servers - Google Patents

Abstract

The image processing method includes a step of acquiring a subject image obtained by capturing an image of the subject (S1), a step of dividing the subject image into a plurality of divided images (S4), and at least a part of the plurality of divided images. The division auxiliary information includes a step (S5) in which the calculated feature amount represents the appearance feature of the subject and a step (S6) in which the division auxiliary information is created based on the feature amount, and the division auxiliary information is the subject image. This is information used to determine the division method.

Description

The present invention relates to an image processing method, an image processing device, a user interface device, an image processing system, and a server.

Conventionally, an image processing technique for recognizing a subject based on the appearance characteristics of the subject in an image is known (see, for example, Patent Document 1). In Patent Document 1, one image is divided into a plurality of small regions, a feature amount representing an appearance feature is calculated for each small area, and a category of each small area is determined based on the feature amount. According to such a method, when a plurality of subjects having different appearance characteristics are included in one image, the plurality of subjects can be recognized separately.

On the other hand, in recent years, in studies using cultured cells, attention has been focused on a technique for estimating parameters (control parameters) indicating cell quality or activity from the appearance characteristics of cells or cell groups. The control parameter is, for example, the proliferation rate, survival rate, or cell number of the cell group.

JP-A-2017-5389

However, the feature amount calculated from the small area may differ depending on the size of the small area. In addition, it may be unclear how large the appearance of the subject correlates with the management parameters.
For example, when a certain number of cells form a characteristic structure, in order to obtain a feature amount that accurately represents the characteristic structure, the size of the small region is set so that the small region contains a certain number of cells. Must be set. If the size of the subregion is too small or too large for the size of a certain number of cells as a whole, the feature amount that accurately represents the characteristic structure cannot be calculated from the subregion. Moreover, there may be no knowledge of how many cells form characteristic structures.

The present invention has been made in view of the above circumstances, and is an image processing method and image processing capable of dividing a subject image into divided images having a size suitable for calculating a feature amount representing the appearance characteristics of the subject. It is an object of the present invention to provide an apparatus, a user interface apparatus, an image processing system, and a server.

In order to achieve the above object, the present invention provides the following means.
One aspect of the present invention includes a step of acquiring a subject image obtained by capturing an image of a subject, a step of dividing the subject image into a plurality of divided images, and calculating at least a part of the features of the plurality of divided images. The division auxiliary information includes a step of representing the appearance feature of the subject and a step of creating division auxiliary information based on the feature amount, and the division auxiliary information is the method of dividing the subject image. It is an image processing method that is information used for determination.

According to this aspect, the subject image is divided into a plurality of divided images, and the feature amount of each divided image is calculated. The feature amount is a quantity representing the appearance feature of the subject in each divided image. Subsequently, the division auxiliary information is created based on the feature amount. Based on such division assistance information, it is possible to determine a method for dividing a subject image that can calculate a feature amount that more accurately represents the appearance characteristics of the subject.

In the above aspect, even if the division auxiliary information includes the distribution map of the feature amount and the plot showing the individual feature amount is displayed on the axis showing the magnitude of the feature amount in the distribution map. Good.
With this configuration, it is possible to determine whether or not the size of the divided image is suitable for calculating the feature amount based on the distribution of the plot on the axis. For example, when a plurality of subjects having different appearance features are included in the subject image and the size of the divided image is appropriate, the feature amount reflecting each appearance feature is calculated, so that the feature amounts are among the feature amounts. The variability is large and the distribution range of the plot on the axis is wide. On the other hand, when the size of the divided image is inappropriate, the variation between the features becomes smaller and the distribution range of the plot on the axis becomes narrower than when the size of the divided image is appropriate.

In the above aspect, the division auxiliary information may include statistical values based on the histogram of the feature amount.
The statistical value of the feature weight histogram is preferably a statistical value representing the width or variance of the histogram. Based on the statistical value of the feature amount histogram, it can be determined whether or not the size of the divided image is suitable for the feature amount calculation. For example, when a plurality of subjects having different appearance features are included in the subject image, when the size of the divided image is appropriate, the feature amount reflecting each appearance feature is calculated, so that the feature amounts are among the feature amounts. The variability is large and the histogram statistics are large. On the other hand, when the size of the divided image is inappropriate, the variation between the features becomes smaller and the statistical value of the histogram becomes smaller than when the size of the divided image is appropriate.

In the above aspect, the division auxiliary information may include the division image associated with each of the plots.
With this configuration, it is possible to provide division auxiliary information that makes it easy to compare the divided image with the feature amount.

In the above aspect, the plot includes a step of acquiring management parameters relating to the internal properties of the subject and a step of classifying the management parameters into one of a plurality of classes according to the size of the subject image. It may be displayed in a manner corresponding to the class of management parameters.
With this configuration, the presence or absence of correlation between the feature amount and the control parameter and the degree of correlation can be recognized from the distribution map.

In the above embodiment, the subject is a cell group to be cultured, and the control parameters are the proliferation rate of the cell group, the survival rate of the cell group after a lapse of a predetermined time after the end of the culture, or after the end of the culture. It may be the number of cells of the cell group after a lapse of a predetermined time.
With this configuration, it is possible to estimate the proliferation rate, survival rate, or cell number, which are the internal properties of the cell group, based on the external characteristics of the cell group in the subject image.

In the above aspect, the step of changing the division method of the subject image to the division method determined based on the division auxiliary information and the step of dividing the subject image into a plurality of divided images by the changed division method. It may be included.
By re-dividing the subject image into a plurality of divided images by the changed division method, it is possible to obtain a divided image capable of calculating the feature amount that more accurately represents the appearance characteristics of the subject.

In the above aspect, the size of the divided image may be changed in the step of changing the dividing method.
With this configuration, when the appearance feature is related to the size of the subject, the feature amount calculated from the divided image of a certain size is compared with the feature amount calculated from the divided image of another size. It is possible to specify the size of the divided image for which the feature amount that more accurately represents the appearance feature is calculated.

In the above aspect, the step of transmitting the division auxiliary information to the user interface device may be included.
With this configuration, the user can confirm the division auxiliary information by using the user interface device.

In the above aspect, the step of accepting the request for changing the division method from the user interface device may be included.
With this configuration, the division method of the subject image can be changed according to the request for changing the division method transmitted from the user interface device by the user.

Another aspect of the present invention includes a processor, the step of acquiring a subject image obtained by capturing a subject, a step of dividing the subject image into a plurality of divided images, and a step of dividing the plurality of divided images. A step of calculating at least a part of the feature amount and the feature amount representing the appearance feature of the subject and a step of creating the division assist information based on the feature amount are executed to perform the division assistance. The information is an image processing device that is information used for determining a method for dividing the subject image.

Another aspect of the present invention is a user interface device that is connected to the image processing device by a communication network and includes a display that displays division auxiliary information received from the image processing device.
Another aspect of the present invention is an image processing system including the image processing device and the user interface device.
Another aspect of the present invention is a server that is connected to the image processing device by a communication network and receives and stores the division auxiliary information.

According to the present invention, there is an effect that the subject image can be divided into divided images having a size suitable for calculating a feature amount representing the appearance characteristics of the subject.

It is an overall block diagram of the image processing system which concerns on one Embodiment of this invention. It is an internal block diagram of the monitoring device, the image processing device, and the UI device of FIG. It is a flowchart which shows the image processing method which concerns on one Embodiment of this invention. It is an example of the original image and the divided image. This is an example of an original image containing a plurality of cell groups having different appearance characteristics. This is an example of the division auxiliary information created by the image processing device and the screen of the UI device. It is another example of the division auxiliary information created by the image processing device and the screen of the UI device. It is a figure explaining the change of the division number of the original image by a slider. It is another example of the division auxiliary information created by the image processing device and the screen of the UI device. It is a flowchart which shows the modification of the image processing method of FIG. It is an overall block diagram of the modification of the image processing system of FIG.

The image processing apparatus, user interface apparatus, image processing system, and server according to the embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the image processing system 100 according to the present embodiment includes a monitoring device 1 that generates an image of a cell group (subject) cultured in the culture vessel 4, and an image of the cell group from the monitoring device 1. It is provided with an image processing device 2 for acquiring an image of a cell group and processing an image of the cell group, and a user interface (UI) device 3 used by the user.

As shown in FIG. 2, the monitoring device 1 includes an imaging device 5, a processor 6, and a communication device 7. The imaging device 5, the processor 6, and the communication device 7 are housed in a sealed box-shaped housing 8 (see FIG. 1). The transparent top plate 8a of the housing 8 is used as a stage on which the culture container 4 is placed. The monitoring device 1 is arranged in an incubator (not shown) together with the culture vessel 4 during the culture period.

The image pickup device 5 has an image pickup device (not shown) such as a CMOS image sensor or a CCD image sensor. The image pickup device 5 takes an image of the inside of the culture vessel 4 on the stage 8a by the image pickup device, and generates an image (subject image) of the cell group.
The processor 6 causes the image pickup device 5 to perform imaging according to a preset schedule or an instruction from the image processing device 2.

The communication device 7 is connected to the communication device 12 (described later) of the image processing device 2 by a communication network 21, and transmits / receives data, information, and signals to and from the image processing device 2. The communication network 21 is, for example, the Internet, an intranet, a LAN (Local Area Network), a WAN (Wide Area Network), or a combination thereof. The communication network 21 may be either wireless or wired. The communication device 7 receives an image from the image pickup device 5 and transmits the image to the communication device 12 of the image processing device 2.

The image processing device 2 includes a memory 10, a processor 11, and a communication device 12. The image processing device 2 is arranged outside the incubator, and receives an image (original image) of a cell group from the monitoring device 1 in the incubator by communication between the communication devices 12 and 7.
An image processing program for causing the processor 11 to execute a predetermined process is stored in the memory 10. The processor 11 reads the image processing program from the memory 10 and executes the processing according to the image processing program to create the division auxiliary information. The image processing and division auxiliary information executed by the processor 11 will be described in detail later.

The communication device 12 is connected to the communication device 16 (described later) of the UI device 3 by a communication network 22, and transmits / receives data, information, and signals to / from the UI device 3. The communication network 22 is, for example, the Internet, an intranet, a LAN, a WAN, or a combination thereof, and may be wireless or wired. The communication device 12 transmits the division auxiliary information created by the processor 11 to the communication device 16 of the UI device 3.

The UI device 3 is a device used by a user to send and receive data, information, and signals to and from the image processing device 2, for example, a general-purpose tablet computer. The UI device 3 includes a display 14, a processor 15, and a communication device 16.
Dedicated application software for displaying the division auxiliary information received from the image processing device 2 is installed in the UI device 3. As will be described later, the user can use this application software to display the division auxiliary information on the display 14 and send a request for changing the division auxiliary information to the image processing device 2.

As shown in FIG. 1, the image processing system 100 may include a plurality of sets of monitoring device 1, image processing device 2, and UI device 3.
Further, the system 100 may include a server 30 connected to a plurality of image processing devices 2 by a communication network 23. The communication network 23 is, for example, the Internet, an intranet, a LAN, a WAN, or a combination thereof, and may be wireless or wired. The server 30 is, for example, a cloud server on the Internet or a computer installed at an arbitrary location.

The server 30 receives and stores the division auxiliary information from the plurality of image processing devices 2. The server 30 may transmit the division auxiliary information received from one image processing device 2 to another image processing device 2. The user can use one UI device 3 to receive and display the division auxiliary information created by the plurality of image processing devices 2.

Next, the image processing method executed by the image processing apparatus 2 will be described.
As shown in FIG. 3, the image processing method according to the present embodiment includes step S1 for acquiring the original image, step S2 for acquiring the management parameters of the original image, step S3 for classifying the management parameters, and the original image. A step S4 for dividing the image into a plurality of divided images, a step S5 for calculating the feature amount of the divided image, a step S6 for creating the division auxiliary information based on the feature amount, and a step of transmitting the division auxiliary information to the UI device 3. It includes S7, a step S8 for accepting a change in the division method of the original image, and a step S9 for re-dividing the original image into a plurality of divided images by the changed division method.

In step S1, the original image is input from the monitoring device 1 to the image processing device 2. The image processing device 2 may acquire a single original image, or may acquire a plurality of original images A, B, and C at one time as shown in FIG. In the example of FIG. 4, the original images A, B, and C are images of cell groups in different cultures.

Next, in step S2, the management parameters of the original image are input to the image processing device 2. Management parameters are parameters related to the internal properties of the cell group, such as the quality or activity of the cell group in the original image. For example, the control parameter is the proliferation rate of the cell group, the survival rate of the cell group after the lapse of a predetermined time after the end of the culture, or the number of cells after the lapse of a predetermined time after the end of the culture.

Control parameters are obtained by measuring cell populations during or after culture. For example, when shipping cells produced by culture, factory viability is used as a control parameter. The shipping survival rate is the survival rate of the cell group in the culture vessel 4 after a lapse of a predetermined time after the completion of the culture. The management parameters are input to the UI device 3 by the user, for example, and are transmitted from the UI device 3 to the image processing device 2.

Next, in step S3, the management parameters are classified into one of a plurality of classes according to their size. For example, as shown in FIG. 4, the shipping survival rate of the original image A is classified into the highest class I, the shipping survival rate of the original image B is classified into the middle class II, and the shipping survival rate of the original image C is classified. Survival rate is classified in the lowest class III. The types and classes of management parameters are stored in the memory 10 in association with the original image.
The management parameters acquired and classified by the image processing apparatus 2 in steps S2 and S3 may be only one type or two or more types. One management parameter is given for each type for one original image.

Next, in step S4, the original image is divided into a plurality of divided images by the initially set division method (step S43). Specifically, as shown in FIG. 4, the original image is equally divided into a preset initial division number, and a divided image having the same size initial division number is generated from one original image. In the example of FIG. 4, the initial number of divisions is 4, and the three original images A, B, and C are divided into four divided images A1 to A4, B1 to B4, and C1 to C4, respectively.

In step S4, the type of cell group and the angle of view of the original image may be set (step S41), and the initial number of divisions may be determined based on the type of cell group and the angle of view of the original image (step S42). Information on the type of cell group and the angle of view of the original image is set in the monitoring device 1 by the user at the start of culturing, and is transmitted from the monitoring device 1 to the image processing device 2 together with the original image.

Next, in step S5, the feature amounts of all the divided images generated in step S4 are calculated. The feature amount is an amount indicating a variation in the appearance characteristics of the cell group in each divided image, for example, shape, color, size, or orientation. The feature amount may be a feature amount that is automatically generated by artificial intelligence such as a neural network and cannot be recognized by humans. The feature amount is calculated based on the information contained in the divided image, for example, the brightness, color, or edge direction of each pixel, and may be calculated using the HOG (Histograms of Oriented Gradients) feature amount.

As shown in FIG. 5, differences in appearance characteristics can occur even in cell groups of the same species cultured in the same culture vessel 4. For example, the cell population in region R1 is small, substantially circular, and forms a monolayer structure. The cell population in region R2 is elliptical, stratified, and irregularly oriented. The cell population in region R3 is elongated, forms a monolayer structure, and is oriented in the same direction. As described above, when a plurality of cell groups having different appearance characteristics are included in one original image A, the feature amounts of the plurality of divided images A1, A2, A3, and A4 vary.

In step S5, the feature amount of some divided images may be calculated instead of all the divided images. For example, a predetermined number of divided images may be selected from all the divided images, and the feature amount of only the selected divided images may be calculated.

Next, in step S6, the division auxiliary information used for determining the division method of the original image is created. As shown in FIG. 6, the division auxiliary information 40 includes a one-dimensional distribution map of the feature amount. In the one-dimensional distribution map, an axis 41 showing the size of the feature amount is defined, and a balloon-shaped plot 42 showing the individual feature amount is displayed on the axis 41. The plot 42 may be any shape such as an arrow, a circle or a polygon.

The plot 42 is displayed in a manner corresponding to the class of control parameters. The aspect is, for example, the color of plot 42. In the example of FIG. 6, the difference in the hatching orientation of the plot 42 represents the difference in the display mode. For example, when the shipping survival rates classes I, II, and III of the original images A, B, and C are different from each other, the feature amount plots 42 of the divided images A1 to A4 and the feature amount plots of the divided images B1 to B4. 42 and the plot 42 of the feature amounts of the divided images C1 to C4 are displayed in different colors.

As a plot of the one-dimensional distribution map, divided images A1 to A4, B1 to B4, and C1 to C4 may be used instead of the graphic. Alternatively, as shown in FIG. 7, the division auxiliary information 40 may include divided images A1 to A4, B1 to B4, and C1 to C4 that are visually associated with the plot 42 and displayed.

Next, in step S7, the division auxiliary information 40 is transmitted from the image processing device 2 to the UI device 3 by communication between the communication devices 12 and 16.
Next, in step S8, a request for changing the method of dividing the original image from the UI device 3 is accepted.

The user uses the application software installed in the UI device 3 to display the division auxiliary information 40 received from the image processing device 2 on the display 14. 6 and 7 show an example of a screen displayed on the display 14 of the UI device 3. On the screen, an area 51 for displaying the division auxiliary information 40, an area 52 for displaying the types of management parameters, and a slider 53 for changing the number of divisions of the original image are displayed.

From the division auxiliary information 40 displayed in the area 51, the user can know whether or not the feature amount correlates with the management parameter.
For example, in the examples of FIGS. 6 and 7, the class I plot 42 is unevenly distributed on the side with the small feature amount, and the class III plot 42 is unevenly distributed on the side with the large feature amount. In such a case, it can be determined that the feature amount correlates with the shipping survival rate. On the other hand, if there is no difference in the distribution of the plot 42 between the classes I, II, and III, it can be determined that the feature amount does not correlate with the shipping survival rate.

As shown in FIG. 8, the user can change the method of dividing the original image, specifically, the number of divisions of the original image, by operating the slider 53. By changing the number of divisions, the size of the divided image is changed. When the user wants to change the number of divisions, the user determines the number of divisions different from the initial number of divisions by operating the slider 53, and transmits a request for change to the determined number of divisions from the UI device 3 to the image processing device 2.

In step S8, the image processing device 2 receives the change of the division method for a certain period of time after transmitting the data of the division auxiliary information 40 to the UI device 3. If there is no request from the UI device 3 to change the division method after a certain period of time has elapsed (NO in step S8), the image processing device 2 ends a series of processes. On the other hand, if there is a request from the UI device 3 to change the number of divisions within a certain period of time (YES in step S8), the image processing device 2 then executes step S9.

In step S9, the original image is equally divided into the changed number of divisions determined by the user, and the changed number of divisions is generated.
Next, the feature amount of the divided image generated in step S9 is calculated (step S5), the division auxiliary information is newly created (step S6), and the newly created division auxiliary information is UI from the image processing device 2. It is transmitted to the device 3 (step S7).

When the number of divisions is changed, the size of the divided images changes, the size of the cell group contained in each divided image changes, the feature amount of the divided images changes, and the distribution of the plot 42 in the one-dimensional distribution map changes. To do.

For example, consider the case where the feature amount is an amount representing the variation in the orientation of the cell group.
When the split image contains a single cell population, the features that most clearly represent the variability in orientation are calculated. Therefore, the features of the plurality of divided images vary, and the plot 42 is distributed over a wide range in the one-dimensional distribution map.
On the other hand, when the divided image contains a plurality of cell groups having different orientation characteristics from each other, the feature amount obtained by averaging the variation in the orientation of the plurality of cell groups is calculated. Therefore, the variation between the features of the plurality of divided images becomes small, and the plot 42 is distributed within a narrow range in the one-dimensional distribution map.

According to the present embodiment, the one-dimensional distribution map displayed on the screen of the UI device 3 is updated every time the user transmits a request for changing the number of divisions of the original image to the image processing device 2. As a result, the user can confirm the variation of the feature amount in each division number while changing the division number, and specify the optimum division number for calculating the feature amount that most clearly expresses the appearance characteristics of the cell group. .. In addition, it is possible to accurately know the presence or absence of correlation between the management parameter and the feature amount and the degree of correlation based on the feature amount of the divided image generated by using the optimum number of divisions.

In addition, once the feature amount that correlates with the management parameter is specified, the management parameter of the cell group can be estimated from the image of the cell group whose management parameter is unknown. That is, the original image of the cell group whose management parameter is unknown can be divided by the optimum number of divisions, the feature amount of the generated divided image can be calculated, and the management parameter can be estimated based on the calculated feature amount. ..

When a plurality of types of management parameters are associated with each original image, the types of management parameters associated with the plot 42 may be changed. For example, a desired type of management parameter may be selectable from the pull-down menu of the area 52. When the type of the management parameter is changed, the display mode of the plot 42 of the one-dimensional distribution map displayed on the UI device 3, for example, the color changes.

In the present embodiment, the maximum number of divisions or the minimum size of the divided image selectable by the slider 53 may be determined according to the number of divisions of the cells in the culture vessel 4.
If cells divide N times during the culture period, one cell grows to ^2N cells. The maximum number of divisions of the original image or the minimum size of the divided image may be determined so that one divided image contains at least about ^2N cells. The number of cell divisions N can be estimated from the culture time.

In the present embodiment, the image processing device 2 decides to create a one-dimensional distribution map of the feature amount as the division auxiliary information 40, but instead of this, as shown in FIG. 9, the feature amount 2 A dimensional distribution map may be created.
In this case, in step S5, a first feature amount and a second feature amount representing different appearance features are calculated. In the two-dimensional distribution map, an axis 41a representing the size of the first feature amount and an axis 41b representing the size of the second feature amount are defined.

In the present embodiment, the user determines whether or not to change the number of divisions and determines the number of divisions based on the division auxiliary information 40 displayed on the UI device 3, but instead of this. , The image processing device 2 may determine the change in the number of divisions and determine the number of divisions. That is, as shown in FIG. 10, the image processing apparatus 2 may execute steps S10 to S12 instead of steps S6 to S8.

After step S5, a histogram of the feature amount is created (step S10), and the statistical value of the histogram is calculated as the division auxiliary information (step S11). The statistic is a value that represents the width or variance of the histogram, preferably the half width of the histogram. The full width at half maximum is, for example, the full width at half maximum of a Gaussian curve that approximates a histogram. The full width at half maximum of the histogram is an index showing the degree of averaging of the feature amounts, like the distribution width of the plot 42 in the one-dimensional distribution map, and changes according to the size of the divided image.

When the full width at half maximum is larger than the predetermined threshold value (YES in step S11), the number of divisions is not changed and the process ends. On the other hand, when the half width is equal to or less than a predetermined threshold value (NO in step S11), the number of divisions is changed (step S12), and then step S9 is executed.
In step S12, the number of divisions is preferably changed in the direction in which the half-value width of the histogram becomes large. Further, while repeating the processes of steps S5, S10, S11, S12, and S9, a table in which the number of divisions and the half width of the histogram are associated is created, and the direction of change of the number of divisions and the number of divisions are determined based on the table. It may be decided.
The histogram and the full width at half maximum may be displayed on the UI device 3 each time the number of divisions is changed, or only the final result may be displayed on the UI device 3.

In the present embodiment, it is decided that a plurality of divided images are generated by equally dividing the original image, but the method of dividing the original image is not limited to this, and the original image is divided by another method. May be done.
For example, even if individual cells are extracted from the entire original image, the original image is divided into the smallest units corresponding to one cell, and a predetermined number of adjacent minimum units are combined to generate a divided image. Good. Further, the size of the divided image may be changed by changing the number of combined minimum units.

In the present embodiment, when the preferable initial number of divisions can be predicted from the cell type, culture condition, or control parameter type, in step S4, the cell type, culture condition, and control parameter type can be predicted. The initial number of divisions may be determined based on at least one.
For example, in culturing iPS cells in an adhesive culture system, it has been confirmed that the cell viability correlates with the layered structure of dozens of cells. Therefore, when investigating the correlation between the viability of adherently cultured iPS cells and the feature amount, the initial number of divisions of the original image is set to a value such that each division image contains dozens of cells. May be good. Further, in step S12, the number of divisions may be changed in the direction in which several tens of cells are included in the divided image.

In the present embodiment, the subject is a group of cells to be cultured in the culture vessel 4, but other subjects may be used. For example, the subject is a lesion observed by an endoscope, and the feature amount of the appearance characteristics of the mucous membrane having a high correlation with the malignancy of the lesion may be calculated.

In the present embodiment, in step S1, the image processing device 2 receives the original image from the monitoring device 1, but instead, the image processing device 2 itself produces the cell group in the culture vessel 4. An image may be taken to generate an original image. That is, the image processing device 2 may include the imaging device 5 and the housing 8 and execute both the generation of the original image and the image processing in the incubator.

In this embodiment, as shown in FIG. 11, the above-mentioned image processing may be executed by the server 30. That is, the image processing device according to this embodiment may be realized as a server.
In this case, the server 30 includes a memory and a processor configured in the same manner as the memory 10 and the processor 11, and is connected to the monitoring device 1 and the UI device 3 by the communication network 23. The original image is transmitted from the monitoring device 1 to the server 30, the division auxiliary information is created by the image processing described above in the server 30, and the division auxiliary information is transmitted from the server 30 to the UI device 3.
In order to realize image processing that requires a lot of processing, high processing capacity is required. By letting the server 30, particularly the cloud server, perform the image processing, the image processing system 100 can be simplified and the cost can be reduced.

1 Monitoring device 2 Image processing device 3 User interface device 4 Culture container 5 Imaging device 6, 11, 15 Processor 7, 12, 16 Communication device 8 Housing 8a Top plate 10 Memory 14 Display 21, 22, 22, 23 Communication network 30 Server 40 Division auxiliary information 41, 41a, 41b Axis 42 Plot 100 Image processing system A, B, C Original image A1 to A4, B1 to B4, C1 to C4 Divided image

Claims (14)

The process of acquiring a subject image that captures the subject, and
The process of dividing the subject image into a plurality of divided images, and
A step of calculating at least a part of the feature amount of the plurality of divided images, and the feature amount is an amount representing the appearance feature of the subject.
Including a step of creating division auxiliary information based on the feature amount.
An image processing method in which the division auxiliary information is information used for determining a method for dividing the subject image. The first aspect of the present invention, wherein the division auxiliary information includes a distribution map of the feature amount, and in the distribution map, plots showing the individual feature amounts are displayed on an axis showing the magnitude of the feature amount. Image processing method. The image processing method according to claim 1, wherein the division auxiliary information includes a statistical value based on a histogram of the feature amount. The image processing method according to claim 2, wherein the division auxiliary information includes the division image associated with each plot. The process of acquiring management parameters related to the internal properties of the subject, and
Including the step of classifying the control parameter into one of a plurality of classes according to its size.
The image processing method according to claim 2 or 4, wherein the plot is displayed in a mode corresponding to the class of management parameters of the subject image. The subject is a group of cells to be cultured.
5. The control parameter is the proliferation rate of the cell group, the survival rate of the cell group after the lapse of a predetermined time after the end of the culture, or the number of cells of the cell group after the lapse of a predetermined time after the end of the culture. The image processing method described in. A step of changing the division method of the subject image to a division method determined based on the division auxiliary information, and
The image processing method according to any one of claims 1 to 6, which includes a step of dividing the subject image into a plurality of divided images by a modified division method. The image processing method according to claim 7, wherein the size of the divided image is changed in the step of changing the division method. The image processing method according to claim 7 or 8, further comprising a step of transmitting the division auxiliary information to the user interface device. The image processing method according to claim 9, further comprising a step of receiving a request for changing the division method from the user interface device. Equipped with a processor
The processor
The process of acquiring a subject image that captures the subject, and
The process of dividing the subject image into a plurality of divided images, and
A process in which at least a part of the features of the plurality of divided images is calculated, and the features represent the appearance features of the subject.
The process of creating the division auxiliary information based on the feature amount is executed.
An image processing device in which the division auxiliary information is information used for determining a method for dividing a subject image. A user interface device that is connected to the image processing device according to claim 11 by a communication network and includes a display that displays division auxiliary information received from the image processing device. The image processing apparatus according to claim 11 and
An image processing system including the user interface device according to claim 12. A server that is connected to the image processing device according to claim 11 by a communication network and receives and stores the division auxiliary information.

Images