WO2019176012A1 - Image processing method, image processing device, user interface device, image processing system and server - Google Patents

Abstract

An image processing method including: a step (S1) for acquiring a subject image in which a subject is imaged; a step (S4) for dividing the subject image into a plurality of divided images; a step (S5) for calculating the feature quantity of at least some of the plurality of divided images, the feature quantity being a quantity that represents the external characteristics of the subject; and a step (S6) for creating auxiliary division information on the basis of the feature quantity. The auxiliary division information is used for determining a method for dividing the subject image.

Description

Image processing method, image processing apparatus, user interface apparatus, image processing system, and server

The present invention relates to an image processing method, an image processing device, a user interface device, an image processing system, and a server.

Conventionally, an image processing technique for recognizing a subject based on appearance characteristics of the subject in an image is known (see, for example, Patent Document 1). In Patent Document 1, a single image is divided into a plurality of small areas, a feature amount representing an appearance feature is calculated for each small region, and a category of each small region is determined based on the feature amount. According to such a method, when a plurality of subjects having different appearance features are included in one image, the plurality of subjects can be distinguished and recognized.

On the other hand, in recent years, in research using cultured cells, a technique for estimating a parameter (management parameter) indicating the quality or activity of a cell from the appearance characteristics of the cell or the cell group has attracted attention. The management parameter is, for example, the proliferation rate, survival rate, or cell number of a cell group.

Japanese Unexamined Patent Publication No. 2017-5389

However, the feature amount calculated from the small area may differ depending on the size of the small area. Also, it may be unclear how much the size of the subject's appearance features correlate with the management parameters.
For example, when a certain number of cells form a characteristic structure, in order to obtain a feature value that accurately represents the characteristic structure, the size of the small area is set so that the small area includes a certain number of cells. Need to be set. When the size of the small area is too small or too large for the entire number of cells, a feature amount that accurately represents a characteristic structure cannot be calculated from the small area. Furthermore, there may be no knowledge of how many cells form a characteristic structure.

The present invention has been made in view of the above-described circumstances, and is an image processing method and an image processing method that can divide a subject image into divided images having a size suitable for calculating a feature amount representing an appearance feature of the subject. An object is to provide a device, a user interface device, an image processing system, and a server.

In order to achieve the above object, the present invention provides the following means.
According to one aspect of the present invention, a step of acquiring a subject image obtained by capturing an image of a subject, a step of dividing the subject image into a plurality of divided images, and calculating at least some of the feature amounts of the plurality of divided images. , The feature amount is an amount representing an appearance feature of the subject, and a step of creating division auxiliary information based on the feature amount, wherein the division auxiliary information is a method of dividing the subject image. This is an image processing method that is information used for determination.

According to this aspect, the subject image is divided into a plurality of divided images, and the feature amount of each divided image is calculated. The feature amount is an amount representing the appearance feature of the subject in each divided image. Subsequently, division assistance information is created based on the feature amount. Based on such division auxiliary information, it is possible to determine a subject image division method that can calculate a feature amount that more accurately represents an appearance feature of a subject.

In the above aspect, the division auxiliary information includes a distribution map of the feature quantity, and in the distribution chart, a plot showing the individual feature quantity is displayed on an axis representing the size of the feature quantity. Good.
With this configuration, it is possible to determine whether the size of the divided image is suitable for calculating the feature amount based on the distribution of plots on the axis. For example, when a plurality of subjects having different appearance features are included in the subject image, when the size of the divided image is appropriate, feature amounts reflecting individual appearance features are calculated. The variation increases and the distribution range of the plot on the axis increases. On the other hand, when the size of the divided image is inappropriate, the variation between the feature amounts is smaller than when the size of the divided image is appropriate, and the distribution range of the plot on the axis is narrow.

In the above aspect, the auxiliary division information may include a statistical value based on the histogram of the feature amount.
The statistical value of the histogram of the feature amount is preferably a statistical value representing the width or variance of the histogram. Based on the statistical value of the histogram of the feature amount, it can be determined whether or not the size of the divided image is suitable for calculating the feature amount. For example, when a plurality of subjects having different appearance features are included in the subject image, when the size of the divided image is appropriate, feature amounts reflecting individual appearance features are calculated. The variation increases and the statistical value of the histogram increases. On the other hand, when the size of the divided image is inappropriate, the variation between the feature amounts becomes smaller and the statistical value of the histogram becomes smaller than when the size of the divided image is appropriate.

In the above aspect, the division auxiliary information may include the divided image associated with each plot.
With this configuration, it is possible to provide division auxiliary information that makes it easy to compare a divided image and a feature amount.

In the above aspect, the method includes a step of obtaining a management parameter relating to an internal property of the subject, and a step of classifying the management parameter into one of a plurality of classes according to the size thereof, wherein the plot includes You may display in the aspect corresponding to the class of a management parameter.
With this configuration, the presence / absence of the correlation between the feature quantity and the management parameter and the degree of correlation can be recognized from the distribution map.

In the above aspect, the subject is a group of cells to be cultured, and the management parameter includes a growth rate of the cell group, a survival rate of the cell group at a predetermined time after the end of the culture, or after the end of the culture. May be the number of cells in the cell group when a predetermined time elapses.
This configuration makes it possible to estimate the growth rate, survival rate, or number of cells, which are the internal properties of the cell group, based on the appearance characteristics of the cell group in the subject image.

In the above aspect, the step of changing the subject image division method to a division method determined based on the division auxiliary information, and the step of dividing the subject image into a plurality of divided images by the changed division method. May be included.
By dividing the subject image again into a plurality of divided images by the changed division method, it is possible to obtain a divided image that can calculate a feature amount that more accurately represents the appearance feature of the subject.

In the above aspect, the size of the divided image may be changed in the step of changing the division method.
By this configuration, when the appearance feature is related to the size of the subject, by comparing the feature amount calculated from the divided image of a certain size with the feature amount calculated from the divided image of another size, It is possible to specify the size of the divided image from which the feature amount that more accurately represents the appearance feature is calculated.

In the said aspect, the process of transmitting the said division | segmentation auxiliary | assistant information to a user interface apparatus may be included.
With this configuration, the user can check the division assistance information using the user interface device.

The above aspect may include a step of receiving a request for changing the division method from the user interface device.
With this configuration, it is possible to change the division method of the subject image in accordance with the request for changing the division method transmitted from the user interface device by the user.

Another aspect of the present invention includes a processor, the processor acquiring a subject image obtained by imaging a subject, a step of dividing the subject image into a plurality of divided images, and among the plurality of divided images, Calculating at least a part of the feature amount, and executing the step of generating the division assistance information based on the feature amount, and the step of generating the division assistance information based on the feature amount The information processing apparatus is an image processing apparatus in which the information is information used for determining the subject image division method.

Another aspect of the present invention is a user interface device that includes a display that is connected to the image processing device via a communication network and displays divisional auxiliary information received from the image processing device.
Another aspect of the present invention is an image processing system including the image processing device and the user interface device.
Another aspect of the present invention is a server that is connected to the image processing apparatus via a communication network and receives and stores the division auxiliary information.

According to the present invention, there is an effect that the subject image can be divided into divided images having a size suitable for calculating the feature amount representing the appearance feature of the subject.

1 is an overall configuration diagram of an image processing system according to an embodiment of the present invention. It is an internal block diagram of the monitoring apparatus of FIG. 1, an image processing apparatus, and UI apparatus. 4 is a flowchart illustrating an image processing method according to an embodiment of the present invention. It is an example of an original image and a divided image. It is an example of the original image containing the several cell group from which an external feature differs mutually. It is an example of the division | segmentation auxiliary information produced by the image processing apparatus, and the screen of UI apparatus. 10 is another example of division auxiliary information created by the image processing apparatus and a screen of the UI apparatus. It is a figure explaining the change of the division | segmentation number of the original image by a slider. 10 is another example of division auxiliary information created by the image processing apparatus and a screen of the UI apparatus. It is a flowchart which shows the modification of the image processing method of FIG. It is a whole block diagram of the modification of the image processing system of FIG.

Hereinafter, an image processing apparatus, a user interface apparatus, an image processing system, and a server according to an embodiment of the present invention will be described with reference to the drawings.
As shown in FIG. 1, the image processing system 100 according to the present embodiment includes a monitoring device 1 that generates an image of a cell group (subject) cultured in the culture vessel 4, and an image of the cell group from the monitoring device 1. And a user interface (UI) device 3 used by a user.

As shown in FIG. 2, the monitoring device 1 includes an imaging device 5, a processor 6, and a communication device 7. The imaging device 5, the processor 6, and the communication device 7 are housed in a sealed box-shaped housing 8 (see FIG. 1). The transparent top plate 8a of the housing 8 is used as a stage on which the culture vessel 4 is placed. The monitoring device 1 is arranged in an incubator (not shown) together with the culture vessel 4 during the culture period.

The imaging device 5 has an imaging element (not shown) such as a CMOS image sensor or a CCD image sensor. The imaging device 5 images the inside of the culture vessel 4 on the stage 8a with an imaging element, and generates an image of the cell group (subject image).
The processor 6 causes the imaging device 5 to perform imaging according to a preset schedule or according to an instruction from the image processing apparatus 2.

The communication device 7 is connected to a communication device 12 (described later) of the image processing device 2 through a communication network 21, and transmits / receives data, information, and signals to / from the image processing device 2. The communication network 21 is, for example, the Internet, an intranet, a LAN (Local Area Network), a WAN (Wide Area Network), or a combination thereof. The communication network 21 may be either wireless or wired. The communication device 7 receives an image from the imaging device 5 and transmits the image to the communication device 12 of the image processing device 2.

The image processing device 2 includes a memory 10, a processor 11, and a communication device 12. The image processing device 2 is disposed outside the incubator, and receives an image (original image) of a cell group from the monitoring device 1 in the incubator through communication between the communication devices 12 and 7.
The memory 10 stores an image processing program for causing the processor 11 to execute predetermined processing. The processor 11 reads out the image processing program from the memory 10 and executes processing according to the image processing program, thereby creating the division auxiliary information. The image processing and division auxiliary information executed by the processor 11 will be described in detail later.

The communication device 12 is connected to a communication device 16 (described later) of the UI device 3 via a communication network 22 and transmits / receives data, information, and signals to / from the UI device 3. The communication network 22 is, for example, the Internet, an intranet, a LAN, a WAN, or a combination thereof, and may be either wireless or wired. The communication device 12 transmits the division auxiliary information created by the processor 11 to the communication device 16 of the UI device 3.

The UI device 3 is a device used by a user to transmit / receive data, information, and signals to / from the image processing device 2, for example, a general-purpose tablet computer. The UI device 3 includes a display 14, a processor 15, and a communication device 16.
The UI device 3 is installed with dedicated application software for displaying the division auxiliary information received from the image processing device 2. As will be described later, the user can use this application software to display the division auxiliary information on the display 14 or to send a request for changing the division auxiliary information to the image processing apparatus 2.

As illustrated in FIG. 1, the image processing system 100 may include a plurality of sets of monitoring apparatuses 1, image processing apparatuses 2, and UI apparatuses 3.
Furthermore, the system 100 may include a server 30 connected to a plurality of image processing apparatuses 2 via the communication network 23. The communication network 23 is, for example, the Internet, an intranet, a LAN, a WAN, or a combination thereof, and may be either wireless or wired. The server 30 is, for example, a cloud server on the Internet or a computer installed at an arbitrary location.

The server 30 receives and stores the divisional auxiliary information from the plurality of image processing apparatuses 2. The server 30 may transmit the division auxiliary information received from one image processing apparatus 2 to another image processing apparatus 2. The user can receive and display division auxiliary information created by a plurality of image processing apparatuses 2 using one UI apparatus 3.

Next, an image processing method executed by the image processing apparatus 2 will be described.
As shown in FIG. 3, the image processing method according to the present embodiment includes a step S1 for obtaining an original image, a step S2 for obtaining a management parameter of the original image, a step S3 for classifying the management parameter, and an original image. Are divided into a plurality of divided images, step S5 is to calculate the feature amount of the divided image, step S6 is to create division auxiliary information based on the feature amount, and step is to transmit the division auxiliary information to the UI device 3. S7, step S8 which receives the change of the division | segmentation method of an original image, and step S9 which again divides | segments an original image into a some divided image with the changed division | segmentation method.

In step S1, the original image is input from the monitoring device 1 to the image processing device 2. The image processing apparatus 2 may acquire a single original image, or may acquire a plurality of original images A, B, and C at a time as shown in FIG. In the example of FIG. 4, the original images A, B, and C are images of cell groups in different cultures.

Next, in step S2, the management parameters of the original image are input to the image processing apparatus 2. The management parameter is a parameter relating to the internal property of the cell group such as the quality or activity of the cell group in the original image. For example, the management parameter is the proliferation rate of the cell group, the survival rate of the cell group when a predetermined time has elapsed after the end of the culture, or the number of cells when the predetermined time has elapsed after the end of the culture.

Management parameters are obtained by measuring cell populations during or after culturing. For example, when shipping cells produced by culture, the shipping survival rate is used as a management parameter. The shipping survival rate is the survival rate of the cell group in the culture container 4 when a predetermined time has elapsed after the end of the culture. The management parameters are input to the UI device 3 by the user, for example, and transmitted from the UI device 3 to the image processing device 2.

Next, in step S3, the management parameter is classified into one of a plurality of classes depending on its size. For example, as shown in FIG. 4, the shipping survival rate of the original image A is classified into the highest class I, the shipping survival rate of the original image B is classified into the middle class II, and the original image C is shipped. Survival is classified as the lowest class III. The type and class of the management parameter are stored in the memory 10 in association with the original image.
The management parameters acquired and classified by the image processing apparatus 2 in steps S2 and S3 may be only one type or two or more types. One management parameter per type is given to one original image.

Next, in step S4, the original image is divided into a plurality of divided images by the initially set division method (step S43). Specifically, as shown in FIG. 4, the original image is equally divided into a preset initial division number, and a divided image having the same initial division number is generated from one original image. In the example of FIG. 4, the initial number of divisions is 4, and three original images A, B, and C are divided into four divided images A1 to A4, B1 to B4, and C1 to C4, respectively.

In step S4, the type of cell group and the angle of view of the original image are set (step S41), and the initial division number may be determined based on the type of cell group and the angle of view of the original image (step S42). Information regarding the type of cell group and the angle of view of the original image is set in the monitoring device 1 by the user at the start of culture, for example, and transmitted from the monitoring device 1 to the image processing device 2 together with the original image.

Next, in step S5, the feature amounts of all the divided images generated in step S4 are calculated. The feature amount is an amount indicating an appearance feature of the cell group in each divided image, for example, variation in shape, color, size, or orientation. The feature amount may be a feature amount that is automatically generated by an artificial intelligence such as a neural network and that cannot be recognized by humans. The feature amount may be calculated based on information included in the divided image, for example, the luminance, color, or edge direction of each pixel, and may be calculated using a HOG (Histograms of Oriented Gradients) feature amount.

As shown in FIG. 5, even in the same type of cell group cultured in the same culture vessel 4, a difference in appearance characteristics may occur. For example, the cell group in the region R1 is a small and substantially circular shape, and forms a single layer structure. The cell group in the region R2 has an elliptical shape, is layered, and is irregularly oriented. The cell group in the region R3 is elongated, forms a single layer structure, and is oriented in the same direction. As described above, when a plurality of cell groups having different appearance features are included in one original image A, the feature amounts of the plurality of divided images A1, A2, A3, and A4 vary.

In step S5, the feature amounts of some divided images instead of all the divided images may be calculated. For example, a predetermined number of divided images may be selected from all the divided images, and the feature amounts of only the selected divided images may be calculated.

Next, in step S6, division auxiliary information used to determine the original image division method is created. As shown in FIG. 6, the division auxiliary information 40 includes a one-dimensional distribution map of feature amounts. In the one-dimensional distribution diagram, an axis 41 representing the size of the feature quantity is defined, and a balloon-like plot 42 showing each feature quantity is displayed on the axis 41. The plot 42 may be any figure such as an arrow, circle, or polygon.

The plot 42 is displayed in a manner corresponding to the management parameter class. An aspect is the color of the plot 42, for example. In the example of FIG. 6, the difference in hatching direction of the plot 42 represents the difference in display mode. For example, when the class I, II, and III of the shipping survival rates of the original images A, B, and C are different from each other, the feature amount plot 42 of the divided images A1 to A4 and the feature amount plot of the divided images B1 to B4 42 and the plots 42 of feature amounts of the divided images C1 to C4 are displayed in mutually different colors.

The divided images A1 to A4, B1 to B4, and C1 to C4 may be used as the plot of the one-dimensional distribution map instead of the figure. Alternatively, as shown in FIG. 7, the divided auxiliary information 40 may include divided images A1 to A4, B1 to B4, and C1 to C4 that are displayed visually associated with the plot 42.

Next, in step S <b> 7, the auxiliary division information 40 is transmitted from the image processing apparatus 2 to the UI apparatus 3 by communication between the communication apparatuses 12 and 16.
In step S8, a request for changing the original image division method from the UI device 3 is received.

The user uses the application software installed in the UI device 3 to display the division auxiliary information 40 received from the image processing device 2 on the display 14. 6 and 7 show an example of a screen displayed on the display 14 of the UI device 3. On the screen, an area 51 for displaying auxiliary division information 40, an area 52 for displaying the type of management parameter, and a slider 53 for changing the number of divisions of the original image are displayed.

The user can know from the auxiliary division information 40 displayed in the area 51 whether or not the feature amount is correlated with the management parameter.
For example, in the example of FIGS. 6 and 7, the class I plot 42 is unevenly distributed on the smaller feature amount side, and the class III plot 42 is unevenly distributed on the larger feature amount side. In such a case, it can be determined that the feature value correlates with the shipping survival rate. On the other hand, if there is no difference in the distribution of the plot 42 between the classes I, II, and III, it can be determined that the feature quantity is not correlated with the shipping survival rate.

As shown in FIG. 8, the user can change the original image dividing method, specifically, the number of original image divisions, by operating the slider 53. The size of the divided image is changed by changing the number of divisions. When the user wants to change the number of divisions, the user determines a number of divisions different from the initial number of divisions by operating the slider 53, and transmits a request to change the determined number of divisions from the UI device 3 to the image processing device 2.

In step S8, the image processing apparatus 2 receives the change of the division method for a certain period after transmitting the data of the auxiliary division information 40 to the UI apparatus 3. If there is no request for changing the division method from the UI device 3 at the time when the fixed period has elapsed (NO in step S8), the image processing device 2 ends the series of processing. On the other hand, when a request for changing the number of divisions is received from the UI device 3 within a certain period (YES in step S8), the image processing device 2 next executes step S9.

In step S9, the original image is equally divided into the changed number of divisions determined by the user, and the divided image having the changed number of divisions is generated.
Next, the feature amount of the divided image generated in step S9 is calculated (step S5), division auxiliary information is newly created (step S6), and the newly created division auxiliary information is sent from the image processing apparatus 2 to the UI. It is transmitted to the device 3 (step S7).

When the number of divisions is changed, the size of the divided image is changed, the size of the cell group included in each divided image is changed, the feature amount of the divided image is changed, and the distribution of the plot 42 in the one-dimensional distribution diagram is changed. To do.

For example, let us consider a case where the feature amount is an amount representing variation in cell group orientation.
When the divided image includes a single cell group, a feature amount that most clearly represents the variation in orientation is calculated. Therefore, variation occurs between the feature amounts of the plurality of divided images, and the plot 42 is distributed over a wide range in the one-dimensional distribution diagram.
On the other hand, when the divided image includes a plurality of cell groups having different orientation characteristics from each other, a feature amount in which variation in orientation of the plurality of cell groups is averaged is calculated. Therefore, the variation between the feature amounts of the plurality of divided images is reduced, and the plot 42 is distributed within a narrow range in the one-dimensional distribution diagram.

According to the present embodiment, each time the user transmits a request for changing the number of divisions of the original image to the image processing apparatus 2, the one-dimensional distribution map displayed on the screen of the UI apparatus 3 is updated. Thereby, the user can confirm the variation of the feature amount in each division number while changing the division number, and can specify the optimum division number for calculating the feature amount that most clearly represents the appearance feature of the cell group. . Further, based on the feature amount of the divided image generated using the optimum number of divisions, the presence / absence of the correlation between the management parameter and the feature amount and the degree of correlation can be accurately known.

In addition, after the feature quantity correlated with the management parameter is specified once, the management parameter of the cell group can be estimated from the image of the cell group whose management parameter is unknown. That is, it is possible to divide an original image of a cell group whose management parameter is unknown by an optimal number of divisions, calculate a feature amount of the generated divided image, and estimate the management parameter based on the calculated feature amount. .

When multiple types of management parameters are associated with each original image, the type of management parameter associated with the plot 42 may be changeable. For example, it may be possible to select a desired type of management parameter from the pull-down menu in the area 52. When the type of the management parameter is changed, the display mode, for example, the color of the plot 42 of the one-dimensional distribution map displayed on the UI device 3 changes.

In the present embodiment, the maximum number of divisions or the minimum size of the divided images that can be selected by the slider 53 may be determined according to the number of divisions of the cells in the culture vessel 4.
If cells divide N times during the culture period, one cell increases to 2 ^N cells. The maximum division number of the original image or the minimum size of the division image may be determined so that at least about ^2N cells are included in one division image. The number N of cell divisions can be estimated from the culture time.

In the present embodiment, the image processing apparatus 2 creates a one-dimensional distribution map of the feature quantity as the auxiliary division information 40. Instead, as shown in FIG. A dimension distribution map may be created.
In this case, in step S5, a first feature amount and a second feature amount representing different appearance features are calculated. In the two-dimensional distribution diagram, an axis 41a representing the size of the first feature value and an axis 41b representing the size of the second feature value are defined.

In this embodiment, the user decides whether to change the number of divisions and determines the number of divisions based on the auxiliary division information 40 displayed on the UI device 3. The image processing apparatus 2 may determine whether to change the division number and determine the division number. That is, as shown in FIG. 10, the image processing apparatus 2 may execute steps S10 to S12 instead of steps S6 to S8.

After step S5, a histogram of the feature amount is created (step S10), and a statistical value of the histogram is calculated as division auxiliary information (step S11). The statistical value is a value representing the width or variance of the histogram, and preferably the half width of the histogram. The half width is, for example, the half width of a Gaussian curve that approximates a histogram. Similar to the distribution width of the plot 42 in the one-dimensional distribution diagram, the half width of the histogram is an index indicating the degree of feature amount averaging, and changes according to the size of the divided image.

If the full width at half maximum is larger than the predetermined threshold (YES in step S11), the number of divisions is not changed and the process ends. On the other hand, when the half width is equal to or smaller than the predetermined threshold (NO in step S11), the number of divisions is changed (step S12), and then step S9 is executed.
In step S12, the number of divisions is preferably changed in a direction in which the half width of the histogram is increased. Further, a table in which the number of divisions and the half width of the histogram are associated with each other is created while repeating the processes of steps S5, S10, S11, S12, and S9, and the change direction and the number of divisions are changed based on the table. It may be determined.
The histogram and the half width may be displayed on the UI device 3 every time the number of divisions is changed, or only the final result may be displayed on the UI device 3.

In this embodiment, a plurality of divided images are generated by equally dividing the original image. However, the original image dividing method is not limited to this, and the original image is divided by other methods. May be.
For example, even if individual cells are extracted from the entire original image, the original image is divided into minimum units corresponding to one cell, and a divided image is generated by combining a predetermined number of minimum units adjacent to each other. Good. Further, the size of the divided image may be changed by changing the number of combined minimum units.

In this embodiment, when a preferable initial division number can be predicted from the cell type, the culture condition, or the management parameter type, in step S4, the cell type, the culture condition, and the management parameter type. The initial division number may be determined based on at least one.
For example, in the culture of iPS cells in an adhesion culture system, it has been confirmed that the cell viability correlates with the stratified structure of several tens of cells. Therefore, when investigating the correlation between the survival rate of iPS cells adherently cultured and the feature amount, the initial division number of the original image is set to a value such that each divided image includes several tens of cells. Also good. In step S12, the number of divisions may be changed in a direction in which several tens of cells are included in the divided image.

In the present embodiment, the subject is a cell group cultured in the culture vessel 4, but may be another subject. For example, the feature amount of the external feature of the mucous membrane, which is a lesion part observed by an endoscope and has a high correlation with the malignancy of the lesion part, may be calculated.

In the present embodiment, in step S1, the image processing apparatus 2 receives the original image from the monitoring apparatus 1. However, instead of this, the image processing apparatus 2 itself selects a cell group in the culture container 4. An original image may be generated by imaging. That is, the image processing apparatus 2 may include the imaging device 5 and the housing 8 and execute both generation of the original image and image processing in the incubator.

In the present embodiment, as shown in FIG. 11, the image processing described above may be executed by the server 30. That is, the image processing apparatus according to the present embodiment may be realized as a server.
In this case, the server 30 includes a memory and a processor configured similarly to the memory 10 and the processor 11, and is connected to the monitoring device 1 and the UI device 3 via the communication network 23. The original image is transmitted from the monitoring device 1 to the server 30, division auxiliary information is created by the above-described image processing in the server 30, and the division auxiliary information is transmitted from the server 30 to the UI device 3.
In order to realize image processing that requires a lot of processing, high processing capability is required. By allowing the server 30, particularly the cloud server, to perform image processing, the image processing system 100 can be simplified and reduced in cost.

DESCRIPTION OF SYMBOLS 1 Monitoring apparatus 2 Image processing apparatus 3 User interface apparatus 4 Culture container 5 Imaging device 6, 11, 15 Processor 7, 12, 16 Communication apparatus 8 Case 8a Top plate 10 Memory 14 Display 21, 22, 22, 23 Communication network 30 Server 40 Split auxiliary information 41, 41a, 41b Axis 42 Plot 100 Image processing systems A, B, C Original images A1-A4, B1-B4, C1-C4 Split images

Claims (14)

1. Acquiring a subject image obtained by imaging the subject;
   Dividing the subject image into a plurality of divided images;
   Calculating a feature amount of at least a part of the plurality of divided images, wherein the feature amount is an amount representing an appearance feature of the subject; and
   Creating division auxiliary information based on the feature amount,
   An image processing method, wherein the division auxiliary information is information used to determine a method for dividing the subject image.
2. The division auxiliary information includes a distribution map of the feature quantity, and a plot showing each of the feature quantities is displayed on an axis representing the size of the feature quantity in the distribution chart. Image processing method.
3. The image processing method according to claim 1, wherein the auxiliary division information includes a statistical value based on a histogram of the feature amount.
4. 3. The image processing method according to claim 2, wherein the auxiliary division information includes the divided images associated with the plots.
5. Obtaining management parameters relating to the internal properties of the subject;
   Classifying the management parameter into any of a plurality of classes according to its size,
   The image processing method according to claim 2, wherein the plot is displayed in a manner corresponding to a management parameter class of the subject image.
6. The subject is a group of cells to be cultured;
   6. The management parameter is a growth rate of the cell group, a survival rate of the cell group at a predetermined time after the end of culture, or a cell number of the cell group at a predetermined time after the end of the culture. An image processing method described in 1.
7. Changing the division method of the subject image to a division method determined based on the division auxiliary information;
   The image processing method according to claim 1, further comprising a step of dividing the subject image into a plurality of divided images by the changed division method.
8. The image processing method according to claim 7, wherein the size of the divided image is changed in the step of changing the division method.
9. 9. The image processing method according to claim 7, further comprising a step of transmitting the division auxiliary information to a user interface device.
10. The image processing method according to claim 9, further comprising a step of receiving a request for changing the division method from the user interface device.
11. With a processor,
    The processor
    Acquiring a subject image obtained by imaging the subject;
    Dividing the subject image into a plurality of divided images;
    Calculating a feature amount of at least a part of the plurality of divided images, wherein the feature amount is an amount representing an appearance feature of the subject; and
    Executing the division auxiliary information based on the feature amount,
    The image processing apparatus, wherein the division auxiliary information is information used to determine a method for dividing the subject image.
12. A user interface device comprising a display that is connected to the image processing device according to claim 11 through a communication network and displays auxiliary division information received from the image processing device.
13. An image processing apparatus according to claim 11;
    An image processing system comprising the user interface device according to claim 12.
14. A server that is connected to the image processing apparatus according to claim 11 through a communication network, and that receives and stores the division auxiliary information.

Images