JP7148776B2 - Colony identification system, colony identification method and colony identification program - Google Patents

Description

The present invention relates to a colony identification system, a colony identification method and a colony identification program.

Techniques for detecting and identifying colonies in which culture media are formed are conventionally known. For example, Patent Documents 1 to 3 disclose techniques for detecting and identifying colonies based on images of culture media in which colonies are formed.

Japanese Patent No. 5850205 Japanese Unexamined Patent Application Publication No. 2011-212013 JP 2012-135240 A

The type of microorganisms that form colonies on a single medium is not limited to one type. In addition, the shape, size, color, etc. of the colonies of microorganisms are diverse. For this reason, in an image that reproduces the normal state seen by humans (image taken with a camera that detects visible light when irradiated with visible light), colonies of different types of microorganisms are captured as similar images. may be Therefore, in the conventional technology, it has been difficult to identify colonies of microorganisms with high accuracy.
The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique that increases the possibility of identifying microorganisms forming colonies with high accuracy.

In order to achieve the above object, a colony identification system acquires a visible light irradiation image, which is an image of a colony of microorganisms formed in a medium that is photographed while the colony is irradiated with visible light. an image acquisition unit, a non-visible light irradiation image acquisition unit that acquires a non-visible light irradiation image that is an image of the colony photographed while the colony is irradiated with non-visible light, and the visible light irradiation image. and a colony identification unit that identifies the microorganisms forming the colony based on the invisible light irradiation image.

That is, in the colony identification system, an image is obtained in each of a visible light irradiation image photographed while the colony is irradiated with visible light and an invisible light irradiation image photographed while the colony is irradiated with invisible light. to shoot. Then, in the colony identification system, microorganisms forming colonies are identified based on the visible light irradiation image and the non-visible light irradiation image. When non-visible light is applied to microorganisms, it may be possible to perform measurements different from visible light, such as fluorescence output from the microorganisms and contrast that cannot be obtained with visible light. Therefore, according to the colony identification system, it is possible to increase the possibility that microorganisms forming colonies can be identified with high accuracy, compared to a configuration that identifies microorganisms based only on images irradiated with visible light.

1A and 1B are diagrams showing images captured by the colony identification system. 1 is a block diagram of a colony identification system; FIG. FIG. 4 is a diagram for explaining teacher data; It is a figure which shows the model of learning object. It is a figure which shows the model of learning object. It is a figure explaining the output value from a model. 4 is a flowchart of machine learning processing; It is a flowchart of colony identification processing.

Here, embodiments of the present invention will be described according to the following order.
(1) Configuration of colony identification system:
(2) Machine learning processing:
(3) Colony identification treatment:
(4) Other embodiments:

(1) Configuration of colony identification system:
1A and 1B are diagrams showing a schematic configuration of the colony identification system 10. FIG. In these figures, the elements of the colony identification system 10 are indicated by numerals, and the members used together with the colony identification system 10 for photographing, etc. are indicated by alphabetic characters. In this embodiment, the object to be imaged by the colony identification system 10 is the plate medium M formed on the petri dish S. As shown in FIG. That is, in the present embodiment, colonies of microorganisms are formed in one or more portions of the plate medium M, and the plate medium M containing the colonies is photographed.

In this embodiment, the petri dish S is placed on a transparent plate T, which is a plate-like member that transmits light. The transparent plate T is supported by a support (not shown) so that the widest surface is parallel to the horizontal plane. Further, in the present embodiment, a reflector R, which is a plate-shaped member that reflects light, is arranged at a position separated from the transparent plate T by a predetermined distance. The widest surfaces of the reflector R and the transparent plate T are oriented parallel to each other. For example, it is realized by a configuration in which the reflector R is placed on a stand and the transparent plate T is supported at a position separated from the stand by a certain distance.

In addition, the reflecting plate R should be able to reflect light, and examples thereof include white paper. Also, in this embodiment, the horizontal plane is regarded as the xy plane, and the vertically upward direction is regarded as the z-axis direction. Furthermore, in this embodiment, a direction in the horizontal plane is defined as the x-direction, and a direction perpendicular to the x-direction is defined as the y-direction.

In this embodiment, a visible light camera 41 and a UV camera 42 are arranged above the petri dish S placed on the transparent plate T. As shown in FIG. That is, the visible light camera 41 and the UV camera 42 are supported by a supporting portion (not shown) so that the plate medium M in the petri dish S is included in the field of view. The visible light camera 41 is a camera with an area sensor that detects the intensity of visible light (400 nm to 750 nm). On the other hand, in the present embodiment, ultraviolet rays (hereinafter referred to as UV) are used as invisible light, and the UV camera 42 is a camera equipped with an area sensor that detects the intensity of invisible light (400 nm or less).

The visible light camera 41 and UV camera 42 may be an integrated camera or separate cameras. In the colony identification system 10 according to this embodiment, the same petri dish S is photographed by the visible light camera 41 and the UV camera 42, respectively. Therefore, when the visible light camera 41 and the UV camera 42 are separate cameras, the positions of the respective cameras may be moved without moving the petri dish S, and photographing may be performed with each camera. The position of the petri dish S may be moved, and photography may be performed with each camera without moving the position of each camera.

In the present embodiment, the visible light source 41a and the UV light source 42a are arranged at two locations in the horizontal direction of the widest surface of the transparent plate T. As shown in FIG. In this embodiment, the visible light source 41a and the UV light source 42a are provided in a common housing, and the visible light source 41a outputs visible light (400 nm to 750 nm) when a voltage is applied. In this embodiment, the UV light source 42a outputs light in a wavelength range of UVA (eg, 320 nm to 400 nm), which is ultraviolet rays.

In this embodiment, the visible light source 41a and the UV light source 42a are provided in a housing that is long in one direction, the longitudinal direction of which is parallel to the zy plane and in both the z and y directions. leaning The visible light source 41a and the UV light source 42a are arranged so that the longitudinal direction of the housing extends from above the transparent plate T to below the transparent plate T as shown in FIG. 1B. In this embodiment, the visible light source 41a and the UV light source 42a are present over substantially the entire length of the housing inside.

Furthermore, in the visible light source 41a and the UV light source 42a, a light output hole 44 is formed on one surface of the housing (see FIG. 1B), and light is output from the output hole 44, but the light is output from the other surface of the housing. No light is emitted from The visible light source 41a and the UV light source 42a are arranged on both sides of the transparent plate T so that the output holes 44 of the housing face the direction of the transparent plate T (to face each other). Therefore, in this embodiment, the petri dish S is illuminated by light sources present above, below and to the sides. In this embodiment, the visible light source 41a and the UV light source 42a are provided in a common housing. is not output.

Light reaching the plate medium M from above the petri dish S can be reflected (including scattering) by colonies formed on the plate medium M. In addition, since the reflector R is provided below the transparent plate T, when the light directed vertically downward is reflected by the reflector R, the reflected light passes through the transparent plate T and reaches the plate culture medium M in the petri dish S. can reach. Transmitted light that has passed through the plate medium M can pass through colonies formed on the plate medium M. Therefore, in this embodiment, when the colony is illuminated by the visible light source 41a and the UV light source 42a provided at two locations and the image of the colony is captured by the visible light camera 41 and the UV camera 42, the reflected light reflected by the colony and An image can be taken based on the transmitted light through the colony.

The image formed by the light reflected by the colony is an image mainly reflecting characteristics such as the shape formed on the surface by the colony. Therefore, according to the image, it is possible to perform identification based on characteristics such as the shape of the colonies formed on the surface (for example, characteristics formed by aerial mycelium or spores). On the other hand, colonies formed on the plate medium M can reach the inside of the plate medium M as well. For example, in the case of actinomycetes, basal hyphae can be formed inside the plate medium M. Transmitted light that has passed through a colony can pass through both the base mycelia and the aerial mycelia above it, so according to the image formed by the transmitted light, the shape that the colony formed on the surface and inside the plate medium M It is possible to perform identification based on characteristics such as.

In addition, in this embodiment, the visible light source 41a and the UV light source 42a are configured integrally, but of course, individual light sources may be used. Also, the number of light sources is not limited. When light sources with individual housings are used for each type of light, one light source is placed to irradiate light and an image is taken, and then the other light source is placed to irradiate light and an image is taken. is done.

In this embodiment, the visible light camera 41 , the UV camera 42 , the visible light source 41 a and the UV light source 42 a are controlled by the controller 20 . FIG. 2 is a block diagram showing the configuration of the colony identification system 10. As shown in FIG.

The colony identification system 10 includes a control section 20, a display section 43, and a storage medium 30 in addition to the above-described visible light camera 41, UV camera 42, visible light source 41a, and UV light source 42a. The control unit 20 includes a CPU, RAM, ROM, and GPU (not shown), and can execute various programs stored in the storage medium 30 or the like. The control unit 20, the storage medium 30, and the display unit 43 may be configured by an integrated computer, or at least part of them may be separate devices, and may be configured to be connected by a USB cable or the like.

In this embodiment, the control section 20 can execute a colony identification program 21 and a machine learning program 22 . The colony identification program 21 is a program that causes the control unit 20 to execute a function of identifying microorganisms forming colonies based on images captured by the visible light camera 41 and the UV camera 42 . When the colony identification program 21 is executed, the control unit 20 functions as a visible light irradiation image acquisition unit 21a, a non-visible light irradiation image acquisition unit 21b, and a colony identification unit 21c.

In this embodiment, the identification is performed based on the results of previously performed machine learning. The machine learning program 22 is a program that causes the control unit 20 to execute a function of machine learning a model for estimating the types of microorganisms forming colonies based on the images captured by the visible light camera 41 and the UV camera 42. is. When the machine learning program 22 is executed, the control section 20 functions as a machine learning section 22a.

(2) Machine learning processing:
In this embodiment, the machine learning process is a process of optimizing a training model forming a neural network. Here, the model is information indicating a formula for deriving a correspondence relationship between the data to be estimated and the data of the estimation result. is the subject of estimation. Teacher data 30 a is recorded in advance in the storage medium 30 in order to perform machine learning processing for optimizing the model for performing this estimation by learning. The control unit 20 optimizes the training model of the neural network based on the teacher data 30a, and stores the optimized model in the storage medium 30 as a trained model 30b.

Machine learning may be performed by various methods, but here an example in which machine learning is performed by YOLO (You Only Look Once) will be described. In order to perform machine learning by YOLO, the teacher data 30a is defined in a format that enables machine learning by YOLO. Specifically, the training data 30a includes images captured by the visible light camera 41 and the UV camera 42, the position of a rectangle (bounding box) surrounding the colony contained in each image, the size of the rectangle, and the This is data in which the probability that microorganisms form a colony is associated with the identification result of microorganisms.

FIG. 3 is a diagram for explaining the teacher data 30a. FIG. 3 shows an image of a petri dish S in which a plurality of colonies are formed on a plate medium M by a plurality of microorganisms. In this example, the microorganisms forming the colonies are actinomycetes, bacteria other than actinomycetes, and fungi, and an example of configuration for identifying these three types will be described below. In FIG. 3, each colony is shown with a rectangle surrounding the colony. In this embodiment, the plate medium M is a medium capable of cultivating actinomycetes.

That is, in the present embodiment, the purpose is to isolate actinomycetes in order to study whether they are useful for various purposes (for example, medicine, agricultural chemicals, etc.). However, in the plate medium M capable of cultivating actinomycetes, bacteria and fungi other than actinomycetes can be cultured. Also, if bacteria and fungi can be isolated, it is possible to study these microorganisms. Therefore, in the present embodiment, in order to be able to identify these actinomycetes, bacteria, and molds, teacher data 30a is defined in advance in which these identification results and images are associated with each other.

YOLO is configured to divide an image into a plurality of grids and identify colonies present in each grid. Also, the identification in each grid is performed based on the rectangles described above. Therefore, in the present embodiment, a rectangle is defined in each grid, and for each rectangle, the position of the rectangle, the size of the rectangle, the probability that microorganisms form a colony in the rectangle, and the identification result of microorganisms are specified in advance. be. Then, teacher data 30a is defined by associating the position of each rectangle, the size of the rectangle, the probability that microorganisms form a colony in the rectangle, and the identification result of microorganisms with the image of petri dish S. .

In the teaching data 30a, the identification results of microorganisms are specified in advance, so the probability that microorganisms form a colony in each rectangle is 0 or 1 (1 is 100%). is 1 and the rest are 0. For example, assume that there is a colony of actinomycetes on a certain grid, and that a rectangle surrounding the colony has X ₁ and Y ₁ positions, W ₁ width and H ₁ height.

In this case, the format of the information on the rectangle is (X coordinate of rectangle, Y coordinate of rectangle, width of rectangle, height of rectangle, probability that microorganisms form a colony in the rectangle, identified microorganism is actinomycete flag indicating that the identified microorganism is bacteria, flag indicating that the identified microorganism is mold), the rectangular data in the grid is (X ₁ , Y ₁ , W ₁ , H ₁ , 1, 1, 0, 0) are associated. _In this embodiment, the width W1 and the height H1 are defined as relative values with respect to the width and height of _a reference rectangle prepared in advance. may be defined by the number of pixels or the like indicating

One or more reference rectangles may be provided, but in this embodiment, five reference rectangles are provided in advance. That is, five types of rectangles having different lengths in the X direction and Y direction are defined in advance and used as reference rectangles. Therefore, in the present embodiment, rectangle information is defined for each of the five types of reference rectangles. That is, the teacher data 30a is defined by associating rectangle information with respect to five types of reference rectangles for each of the grids described above. In FIG. 3, rectangles with a probability of 1 that microorganisms form colonies in the teacher data 30a are extracted and indicated by white lines.

The teacher data 30a as described above is prepared in advance before machine learning is performed. When the teacher data 30a is prepared, machine learning can be performed using the teacher data 30a. A model can be defined in various ways as long as it converts input data into output data. In this embodiment, a model is constructed in advance that inputs an image containing colonies, passes through a plurality of convolution layers and a fully connected layer, and outputs rectangular information for each grid.

In this embodiment, a visible light irradiation image and an invisible light irradiation image are input data. Here, the visible light irradiation image is an image obtained by photographing the colony of microorganisms formed in the culture medium while the colony is irradiated with visible light. That is, the visible light irradiation image is an image of the petri dish S photographed by the visible light camera 41 while the petri dish S is irradiated with the output light of the visible light source 41a.

In addition, the invisible light irradiation image is an image of the colony photographed while the colony is irradiated with invisible light. In this embodiment, the invisible light is ultraviolet (UV) light, and images are captured by both the visible light camera 41 and the UV camera 42 when the invisible light is irradiated. That is, an image of the petri dish S photographed by the visible light camera 41 and the UV camera 42 in a state where the petri dish S is irradiated with the output light of the UV light source 42a is the non-visible light irradiation image. As described above, in this embodiment, regardless of the frequency of light that can be detected by a camera, an image is irradiated with visible light based on whether the light output from the light source is visible light or non-visible light. It is called either an image or an invisible light irradiation image. Table 1 shows the frequency range of the light source and the frequency range of the camera for the visible light irradiation image and the invisible light irradiation image in this embodiment.

Note that the visible light irradiation image is an image close to the image when a person visually recognizes the petri dish S, but the non-visible light irradiation image is different from the image when a person visually recognizes the petri dish S. In other words, features that are different from features normally visually recognized by humans are analyzed. When the petri dish S irradiated with UV light is photographed by the visible light camera 41, light of the same frequency as the UV light is not photographed, but when the microorganisms emit fluorescence due to the UV light, they are photographed. Therefore, according to the non-visible light irradiation image captured by the visible light camera 41, it is possible to analyze the fluorescence characteristics. In this way, according to the configuration in which the object of analysis is an image captured by a camera that detects light in a wavelength range different from the wavelength range of light irradiated to the colony, a reaction different from simple reflection or transmission of light (fluorescence colonies exhibiting output, etc.) can be effectively imaged and identified.

When the Petri dish S irradiated with UV light is photographed by the UV camera 42, the light of the same frequency as the visible light visually recognized by humans is not photographed, but it is possible to visualize and analyze the characteristics of microorganisms under illumination with UV light. be. Note that the visible light camera 41 outputs the intensity of each color channel of RGB (R: red, G: grain, B: blue) for each pixel. The data are 3-channel data.

On the other hand, in this embodiment, the UV camera 42 photographs the petri dish S illuminated by the UV light source 42a that outputs UVA (eg, 320 nm to 400 nm). In this case, the wavelength range of ultraviolet rays detected by the UV camera 42 is also the same as that of UVA, and is narrower than the wavelength range of visible light. Therefore, in this embodiment, it is assumed that the output from the UV camera 42 can be represented by one channel. That is, the image data representing the image captured by the UV camera 42 is 1-channel data.

As described above, the image data representing the image captured by the visible light camera 41 in this embodiment is 3-channel data. The visible light camera 41 takes an image while the petri dish S is illuminated by the visible light source 41a and the UV light source 42a, respectively, and the images are a visible light irradiation image and an invisible light irradiation image, respectively. Therefore, both the visible light irradiation image and the invisible light irradiation image photographed by the visible light camera 41 are 3-channel data (see Table 1). On the other hand, the image data representing the image captured by the UV camera 42 is 1-channel data. The UV camera 42 takes an image while the petri dish S is illuminated by the UV light source 42a, and outputs an invisible light irradiation image. Therefore, the invisible light irradiation image captured by the UV camera 42 has one channel (see Table 1).

In this embodiment, the input data are a 3-channel visible light irradiation image and an invisible light irradiation image, and a 1-channel invisible light irradiation image. Therefore, a total of 7-channel images obtained by photographing the petri dish S serve as input data. In this embodiment, instead of being processed by a CNN (Convolutional Neural Network) that convolves all 7 channels, a 3-channel visible light irradiation image, a 3-channel invisible light irradiation image, and a 1-channel invisible light irradiation image are processed. Each of the illuminated images is processed in parallel by a separate CNN. Finally, output data is obtained by constructing a neural network that synthesizes individual results.

4 and 5 are explanatory diagrams for explaining examples of neural networks. In these figures, the left end of FIG. 4 is the input layer, and the layer becomes deeper as it moves to the right of the drawing. The layer described on the right end of FIG. 4 and the layer described on the left end of FIG. 5 are the same, and the right end of FIG. 5 is the output layer. In FIGS. 4 and 5, rectangular parallelepipeds schematically show the format of input data in each layer. In FIG. 4, an example of a visible light irradiation image and an invisible light irradiation image of the petri dish S is shown at the left end. That is, a 3-channel visible light irradiation image, a 3-channel non-visible light irradiation image, and a 1-channel non-visible light irradiation image are shown.

In this example, the size of these images is 1024 pixels in each direction. The input layer shown in FIG. 4 schematically shows such an image. That is, since the visible light irradiation image in the uppermost stage is data of 1024 pixels in length and width and 3 channels, it is represented by a rectangular parallelepiped having a width of 1024 and a depth of 3. Since the invisible light irradiation image in the middle row is also data of 1024 pixels in length and width and 3 channels, it is represented by a rectangular parallelepiped with a width of 1024 and a depth of 3. On the other hand, since the invisible light irradiation image in the lower stage is data of 1024 pixels in length and width and 1 channel, it is represented by a rectangular parallelepiped with a width of 1024 and a depth of 1. In the examples shown in FIGS. 4 and 5, the length scales of the rectangular parallelepipeds are not always the same in the vertical, horizontal, and depth directions.

In FIG. 4, after each image data is input to the input layer, it passes through CNN, that is, through a convolution operation with a predetermined size and number of filters, an operation with an activation function, and an operation with a pooling layer. An example of conversion to x32 output values is shown. In FIG. 4, it is then converted into 256×256×64 output values through a convolution operation using a predetermined size and number of filters, an operation using an activation function, and an operation using a pooling layer. It is converted into 32×32×1024 output values through calculation.

Such CNN processing is individually performed for each of the 3-channel visible light irradiation image, the 3-channel non-visible light irradiation image, and the 1-channel non-visible light irradiation image. For this reason, in the present embodiment, the control unit 20 includes at least three GPUs, and a three-channel visible light irradiation image, a three-channel non-visible light irradiation image, and a one-channel non-visible light irradiation image. are performed on different GPUs. Note that the above CNN calculations may be executed collectively for 7 channels (or in the state of 9 channels), but in order to execute such calculations, a GPU with a very large capacity memory is required. However, as in the present embodiment, if three GPUs are used to process each of 3 channels, 3 channels, and 1 channel, the memory capacity can be reduced compared to the case of performing calculations collectively. , computation can be performed using an inexpensive GPU. In addition, learning can be performed at high speed compared to the case of performing calculations collectively.

In this embodiment, the model is constructed so that the output values calculated in parallel as described above are combined, and finally, the information corresponding to the format of the rectangular information in the teacher data 30a is output. It is FIG. 5 shows how 3-channel, 3-channel, and 1-channel images are combined after parallel CNN processing. That is, 32×32×1024 output values obtained by parallel processing by CNN are combined and regarded as 32×32×3072 output values.

Then, this output value is regarded as a state in which 3072 output values are obtained for each of 32×32 grids, and 3072 data of each grid are input values, and 40 output values are obtained. A fully connected layer is defined. As a result, 32×32×40 output values are finally obtained as shown in FIG.

The output value is information according to the format of the rectangular information in the teacher data 30a. FIG. 6 is a diagram showing the structure of the output value. Of the 32×32×40 output values, 32 pieces of information each in the vertical and horizontal directions correspond to each of the 32×32 grids. On the other hand, 40 pieces of information arranged in the depth direction are formed by arranging 5 pieces of 8 pieces of information. That is, the format of the rectangle information is (X coordinate of rectangle, Y coordinate of rectangle, Width of rectangle, Height of rectangle, Probability that microorganisms form a colony in the rectangle, Identified microorganisms are actinomycetes. a flag indicating that the identified microorganisms are bacteria; a flag indicating that the identified microorganisms are fungi), and one rectangle has eight pieces of data.

In this embodiment, since these data are defined for each of the five types of reference rectangles, 40 pieces of information are defined per grid. In FIG. 6, information about a certain rectangle in a certain grid is extracted, and the X coordinate of the rectangle is X, the Y coordinate of the rectangle is Y, the width of the rectangle is W, and the height of the rectangle is H. In FIG. In addition, Po is the probability that a microorganism forms a colony in the rectangle, Pa is a flag indicating that the identified microorganism is actinomycete, Pb is a flag indicating that the identified microorganism is bacteria, and Pm indicates a flag indicating that the microorganisms detected are fungi. In this embodiment, Po indicates the probability that a microorganism forms a colony, and Pa, Pb, and Pm indicate the type of microorganism. It can be said that it is a model that outputs the probability of forming a colony in

These 8 output values are obtained for each of the 5 types of rectangles, resulting in 40 output values. It is a training model in morphology. That is, in the above model, by machine-learning variable parameters (for example, filter weights, etc.), it is possible to obtain a learned model 30b that distinguishes the presence of microorganisms and the types of microorganisms. Of course, the models shown in FIGS. 4 to 6 are only examples, and the size, number and type of filters, types of activation functions, types of padding and strides, types and presence/absence of pooling layers, presence/absence of fully connected layers, etc. good to be changed.

In this embodiment, machine learning processing is executed based on the above model. FIG. 7 is a flowchart showing machine learning processing. Machine learning processing is pre-performed before microorganism identification is performed. In addition, teacher data 30a is prepared in advance before the machine learning process is performed. When the machine learning process is started, the control unit 20 acquires a training model using the function of the machine learning unit 22a (step S100). In the present embodiment, the control unit 20 uses an image of the petri dish S as an input value, and a training model (a filter or activation model indicating a model) whose output value is information on a rectangle superimposed on the image in the petri dish S as shown in FIG. function).

Next, the control unit 20 acquires the teacher data 30a by the function of the machine learning unit 22a (step S105). In the present embodiment, the training data 30a associates rectangle information (rectangle size, identification result, etc.) with visible light irradiation images and non-visible light irradiation images as shown in Table 1 in which petri dish S is photographed. It is information that has been generated in advance.

Next, the control unit 20 acquires test data using the function of the machine learning unit 22a (step S110). In the present embodiment, a part of the teacher data 30a is extracted and used as test data for confirming whether learning has been generalized. Note that test data is not used for machine learning.

Next, the control unit 20 determines an initial value using the function of the machine learning unit 22a (step S115). That is, the control unit 20 gives initial values to the variable parameters to be learned (such as filter weights and biases) in the training model acquired in step S100. The initial value can be determined in various ways. Of course, the initial values may be adjusted so that the parameters are optimized in the process of learning, or learned parameters may be acquired from various databases or the like and used.

Next, the control unit 20 performs learning using the function of the machine learning unit 22a (step S120). That is, the control unit 20 inputs the image of the teacher data 30a acquired in step S105 to the training model acquired in step S100, and outputs information indicating the identification result. At this time, the control unit 20 inputs the 3-channel visible light irradiation image, the 3-channel non-visible light irradiation image, and the 1-channel non-visible light irradiation image to each of the three GPUs.

Each GPU computes 32×32×1024 output values according to the model shown in FIG. After that, the GPU or CPU combines these output values to obtain 32×32×40 output values shown in FIG. When the final output value is obtained, the control unit 20 specifies the error by a loss function indicating the error between the output value and the information indicating the identification result indicated by the teacher data 30a.

That is, in the teacher data 30a, 8 rectangles are defined for each of the 5 types of reference rectangles, for a total of 40 rectangles, and are associated with 32×32 grids. On the other hand, 40 output values from the model are obtained for 32×32 grids as shown in FIG. be.

Focusing on a specific rectangle in a certain grid, in both the teacher data 30a and the output value, the information of the rectangle is the position (X, Y), the size (W, H), the probability that the microorganisms are forming a colony ( Po), and is represented by the identification results of microorganisms (Pa, Pb, Pm). In this way, since the information associated with the teacher data 30a and the output value by the model correspond, the control section 20 identifies the error based on the loss function indicating the difference between the corresponding information. Once the loss function E is obtained, the control unit 20 updates the parameters by a predetermined optimization algorithm, such as stochastic gradient descent. That is, the control unit 20 repeats the process of updating the parameters based on the differentiation of the loss function E by the parameters a predetermined number of times.

After the parameters have been updated a predetermined number of times as described above, the control unit 20 determines whether or not the generalization of the training model has been completed (step S125). That is, the control unit 20 inputs the test data acquired in step S110 to the training model and acquires an output value indicating the microorganism identification result. Then, the control unit 20 obtains the number of matches between the output identification result and the identification result associated with the test data, and divides the number by the number of samples to obtain the estimation accuracy. In this embodiment, the control unit 20 determines that the generalization is completed when the estimation accuracy is equal to or higher than the threshold.

In addition to the evaluation of the generalization performance, verification of the validity of the hyperparameters may be performed. That is, in a configuration in which hyperparameters that are variable quantities other than variable parameters to be learned, such as the number of nodes, are tuned, the control unit 20 verifies the validity of the hyperparameters based on the verification data. Also good. The verification data may be obtained by extracting the verification data in advance from the teacher data 30a by the same process as in step S110 and securing it as data that is not used for training.

If it is determined in step S125 that the generalization of the training model has not been completed, the control unit 20 repeats step S120. That is, the process of updating the variable parameters to be further learned is performed. On the other hand, when it is determined in step S125 that the generalization of the training model has been completed, the control unit 20 records the learned model (step S130). That is, the control unit 20 records the training model in the storage medium 30 as the learned model 30b.

(3) Colony identification treatment:
Next, the colony identification process performed for unidentified colonies formed on the plate medium M will be described based on the flowchart shown in FIG. When the petri dish S containing the plate medium M on which the colonies are formed is prepared, the petri dish S is set at a predetermined position on the transparent plate T within the visual field of the visible light camera 41 . In this state, when the user instructs the start of colony identification processing using an operation unit such as a mouse or a keyboard (not shown), the control unit 20 starts executing the colony identification program 21 .

When execution of the colony identification program 21 is started, the control unit 20 irradiates the petri dish S with visible light using the function of the visible light irradiation image acquiring unit 21a (step S200). That is, the control unit 20 outputs a control signal to the visible light source 41a to turn on the visible light source 41a. At this time, the UV light source 42a is turned off. As a result, the petri dish S is irradiated with visible light, and an image is formed on the imaging element in the visible light camera 41 by the reflected light from the microorganisms forming the colony and the transmitted light from the reflector R transmitted through the microorganisms. state.

Next, the control unit 20 captures a visible light irradiation image with the visible light camera 41 using the function of the visible light irradiation image acquiring unit 21a (step S205). That is, the control unit 20 controls the visible light camera 41 to obtain image data of the petri dish S irradiated with visible light. The captured image data is 3-channel image data, and the control unit 20 records the image data in the RAM or storage medium 30 in association with information indicating that it is a 3-channel visible light irradiation image.

Next, the control unit 20 causes the petri dish S to be irradiated with UV light using the function of the invisible light irradiation image acquisition unit 21b (step S210). That is, the controller 20 outputs a control signal to the UV light source 42a to turn on the UV light source 42a. At this time, the visible light source 41a is turned off. As a result, the petri dish S is irradiated with UV light, and an image is formed on the imaging element in the visible light camera 41 by the reflected light from the microorganisms forming the colony and the transmitted light from the reflector R transmitted through the microorganisms. state.

Next, the control unit 20 captures an invisible light irradiation image with the visible light camera 41 using the function of the invisible light irradiation image acquiring unit 21b (step S215). That is, the control unit 20 controls the visible light camera 41 to acquire image data of the petri dish S irradiated with UV light. The captured image data is 3-channel image data, and the control unit 20 associates the image data with information indicating that it is a 3-channel invisible light irradiation image, and records the information in the RAM or the storage medium 30. .

In this embodiment, the control unit 20 further captures an invisible light irradiation image with the UV camera 42 . For this purpose, the user temporarily turns off the UV light source 42a, removes the visible light camera 41, and fixes the UV camera 42 to a supporting portion (not shown). In this state, the control unit 20 causes the petri dish S to be irradiated with UV light using the function of the invisible light irradiation image acquiring unit 21b (step S216). That is, the controller 20 outputs a control signal to the UV light source 42a to turn on the UV light source 42a. At this time, the visible light source 41a is turned off. As a result, the petri dish S is irradiated with UV light, and an image is formed on the imaging device in the UV camera 42 by the reflected light from the colony-forming microorganisms and the transmitted light from the reflector R transmitted through the microorganisms. state.

Next, the control unit 20 captures an invisible light irradiation image with the UV camera 42 using the function of the invisible light irradiation image acquiring unit 21b (step S217). That is, the control unit 20 controls the UV camera 42 and acquires image data of the petri dish S irradiated with UV light. Since the captured image data is 1-channel image data, the control unit 20 associates the image data with information indicating that it is a 1-channel invisible light irradiation image, and records the data in the RAM or the storage medium 30. do.

Of course, after the UV camera 42 acquires three-channel image data, the information may be compressed to one channel. Moreover, the acquisition order of the visible light irradiation image and the acquisition order of the non-visible light irradiation image is an example, and other orders may be used. The photographing conditions for the image of the Petri dish S to be identified as described above are the same as the photographing conditions for photographing the image of the teacher data 30a.

When the 3-channel visible light irradiation image, the 3-channel non-visible light irradiation image, and the 1-channel non-visible light irradiation image are obtained by the above processing, the control unit 20 performs the functions of the colony identification unit 21c. generates a 7-channel input image (step S220). That is, the control unit 20 cuts out an image of the petri dish S having a predetermined size (1024×1024 pixels in the above example) from each image data acquired in steps S205 and S215, and regards the image as an input image. Of course, various image processing may be performed here.

When the input image is obtained, the control unit 20 inputs the input image to the learned model using the function of the colony identification unit 21c (step S225). That is, the control unit 20 acquires the learned model 30b recorded in the storage medium 30, and acquires the output value when the input image is input. As a result, in each of the 32×32 grids, identification results for five types of reference rectangles are obtained.

Next, the control section 20 suppresses the output by the function of the colony identification section 21c (step S230). That is, the control unit 20 performs processing for leaving significant output values as identification results. Suppression of power output may be accomplished in a variety of ways. For example, according to the process of leaving an output value in which the probability Po that a microorganism forms a colony in a rectangle is equal to or greater than a predetermined threshold value (for example, 0.5), a rectangle with a high possibility of forming a microorganism is selected. can be left. In addition, when two rectangles overlap at a predetermined ratio or more, the process of leaving one of the two and not leaving the other prevents detection of the same colony from being indicated by a plurality of rectangles. The ratio of overlapping rectangles can be evaluated, for example, by IOU (Intersection Over Union: logical product (area) of two rectangles/logical sum (area) of two rectangles).

Next, the control section 20 displays the identification result by the function of the colony identifying section 21c (step S235). That is, the control unit 20 controls the display unit 43 to display the identification result based on the output value obtained as a result of the suppression in step S230. The identification result may be displayed in various modes, and in the present embodiment, a mode in which the petri dish S image is superimposed with a rectangle is adopted. That is, when the identification result is displayed, an image as shown in FIG. 3 is displayed on the display section 43 .

According to the present embodiment as described above, by inputting the visible light irradiation image and the invisible light irradiation image of the petri dish S into the learned model 30b and outputting the information indicating the identification result, the colony in any petri dish S It is possible to identify the microorganisms forming the Conventionally, the identification of actinomycetes, bacteria, and fungi has often been performed artificially based on images, requiring many years of experience and expertise. However, according to this embodiment, in which machine learning results are used for identification, even users without many years of experience or specialized knowledge can obtain identification results. That is, it can be said that it is a so-called artificial intelligence (AI) system.

Moreover, even for those with many years of experience and expertise, the task of identifying individual colonies is very time consuming. However, according to this embodiment, in which machine learning results are used for identification, it is possible to identify a large number of colonies in a short period of time.

Furthermore, in the present embodiment, the petri dish S is placed on the transparent plate T, and the reflector R is arranged at a position separated from the transparent plate T by a certain distance. The reflector R is, for example, white paper or the like and has a property of reflecting light, and the transparent plate T allows light to pass therethrough. Although the plate medium M has a dark color, it allows light to pass therethrough. Therefore, in this embodiment, as shown in FIG. 3, an image is taken while light is transmitted through the plate medium M. FIG. For this reason, the periphery of the colony formed on the plate medium M does not become excessively dark (for example, black), and machine learning and processing are performed in a state in which the details of the colony (for example, the shape of the edge, etc.) are reflected in the image. Identification can be done. Therefore, according to the illumination system according to this embodiment, it is possible to improve the identification accuracy.

Furthermore, in the present embodiment, identification is performed based on an invisible light irradiation image of the colony illuminated by the UV light source 42a captured by the visible light camera 41. FIG. Therefore, when actinomycetes or metabolites exhibit fluorescence when irradiated with UV light, which is invisible light, they can be identified based on this fluorescence. Furthermore, in this embodiment, identification is performed based on an invisible light irradiation image captured by the UV camera 42 of the colony illuminated by the UV light source 42a. Therefore, identification is performed based on an image obtained by irradiation with UV light, which is invisible light, that is, an image that is normally invisible to humans. Therefore, it is possible to perform identification based on features beyond the knowledge of those who have many years of experience and specialized knowledge.

(4) Other embodiments:
The above embodiment is an example for carrying out the present invention, and there are various other embodiments as long as the microorganisms forming colonies are identified based on the visible light irradiation image and the non-visible light irradiation image. can be adopted. For example, in the above-described embodiment, both the 3-channel invisible light irradiation image captured by the visible light camera 41 and the 1-channel invisible light irradiation image captured by the UV camera 42 are used. However, either one may be used.

Also, the colony identification system may be implemented by a plurality of devices, or may be a system in which machine learning is performed by a server and identification is performed by a client. Furthermore, at least a part of the visible light irradiation image acquisition unit 21a, the non-visible light irradiation image acquisition unit 21b, the colony identification unit 21c, and the machine learning unit 22a may exist separately in a plurality of devices. For example, in the colony identification unit 21c, the process of extracting rectangles from the petri dish S and the process of identifying colonies included in the rectangles may be performed by different apparatuses. Of course, some configurations of the above-described embodiments may be omitted, and the order of processing may be changed or omitted. For example, a configuration or the like may be employed in which switching of the light source is performed manually instead of the control unit 20 .

The visible light irradiation image acquisition unit only needs to be able to acquire a visible light irradiation image, which is an image of a colony of microorganisms formed in a culture medium that is photographed while the colony is irradiated with visible light. That is, the colony is irradiated with light in a wavelength range that can be recognized by the human eye, and an image formed by the light output from the colony as a result is acquired as a visible light irradiation image. The light output from the colonies may be reflected light (including scattered light), transmitted light, fluorescence, or light containing these lights. The wavelength range of the light output from the colony may be the wavelength range of visible light or the wavelength range of non-visible light. The former is taken with a camera that can detect visible light, and the latter is taken with a camera that can detect invisible light.

Visible light may be light that can be recognized by human eyes, and for example, light in the wavelength range of 400 nm to 750 nm can be considered visible light. The wavelength range of visible light may be arbitrary, but at least the wavelength range of visible light and the wavelength range of non-visible light include a wavelength range that does not overlap, and the latter is unrecognizable to the human eye. is set to include

Any medium may be used as long as the microorganism to be identified can be cultured, and various media may be employed depending on the microorganism to be identified. Of course, if identification of unknown microorganisms is assumed without limiting identification targets in advance, various culture media may be prepared instead of specific culture media. Furthermore, the medium may be prepared in a container such as a petri dish, and may be used as a plate medium. Also, the medium may be in a state in which a plurality of colonies can be formed.

Microorganisms may be any organisms that are cultured in a medium to form colonies, and include various fungi including actinomycetes (eubacteria, archaea, eukaryotes (algae, protists, fungi, slime molds), etc.). can be, but it can also be other very small organisms. Also, the microorganisms to be identified may be known or unknown.

Visible light may be emitted in various ways. That is, the visible light may be applied to the colonies so that visible light irradiation images can be taken, and the positional relationship between the colonies and the visible light source can be various. Of course, the positional relationship may be variable, and the intensity of visible light may be variable.

The visible light irradiation image may be captured in a state where the intensity of the visible light is a specific intensity, or may be captured in a state where the intensity of the visible light is a plurality of intensities, and various modes may be adopted. Moreover, the image irradiated with visible light may be an image obtained by photographing a colony of microorganisms, as long as at least one colony is included. Of course, it may contain a plurality of colonies. Although the number of channels for expressing the visible light irradiation image is arbitrary, it is preferable that the visible light irradiation image is typically represented by three channels in order to cover the wavelength range of visible light.

In addition, the visible light irradiation image may be a two-dimensional image of the colony in a state where the colony that spreads two-dimensionally is irradiated with visible light. I wish I could. That is, in the colony identification system, it is sufficient that the characteristics of the colony are captured in a two-dimensional image and the microorganisms forming the colony can be identified based on the image.

The invisible light irradiation image acquiring unit only needs to be able to acquire an invisible light irradiation image, which is an image of a colony photographed while the colony is irradiated with invisible light. In other words, it is sufficient if the non-visible light irradiation image can be obtained for the same colony as the colony for which the visible light irradiation image was captured. At this time, the colony is irradiated with light in a wavelength range that cannot be recognized by the human eye, and an image formed by the light emitted from the colony as a result is obtained as an invisible light irradiation image.

The light output from the colonies may be reflected light (including scattered light), transmitted light, fluorescence, or light containing these lights. The wavelength range of the light output from the colony may be the wavelength range of visible light or the wavelength range of non-visible light. The former is taken with a camera that can detect visible light, and the latter is taken with a camera that can detect invisible light.

The invisible light may be light that cannot be recognized by human eyes, and may be ultraviolet light, infrared light, or both ultraviolet light and infrared light. In the case of ultraviolet rays, any of UVA (eg, 320 nm to 400 nm), UVB (eg, 290 nm to 320 nm), UVC (eg, 200 nm to 290 nm) may be used, and multiple bands of multiple wavelength ranges may be included. good. In order to reduce the influence on microorganisms, etc., the longer the wavelength of the ultraviolet rays, the better. For example, UVA may be selected. Similar considerations may be made for IR. Furthermore, when fluorescence is expected to be output from microorganisms, a wavelength for outputting the fluorescence may be selected.

Invisible light may be emitted in various ways. In other words, the invisible light may be applied to the colony so that the invisible light irradiation image can be captured, and the positional relationship between the colony and the invisible light source can be various. Of course, the positional relationship may be variable, and the intensity of the invisible light may be variable.

The invisible light irradiation image may be captured with the invisible light having a specific intensity, or may be captured with a plurality of intensities, and various modes may be employed. Also, the invisible light irradiation image may be an image of colonies of microorganisms, as long as at least one colony is included. Of course, it may contain a plurality of colonies. Any number of channels may be used to express the invisible light irradiation image. When a relatively narrow wavelength range (for example, UVA) is used, for example, it is often possible to express an invisible light irradiation image with one channel, and an invisible light irradiation image can be expressed with a wider wavelength range. In some cases, multiple channels may be used.

In addition, the invisible light irradiation image may be a two-dimensional image of the colony in a state where the colony that spreads at least two-dimensionally is irradiated with invisible light, and a camera capable of photographing a two-dimensional area. It should be taken with That is, in the colony identification system, it is sufficient that the characteristics of the colony are captured in a two-dimensional image and the microorganisms forming the colony can be identified based on the image.

The colony identification unit only needs to be able to identify microorganisms forming colonies based on the visible light irradiation image and the non-visible light irradiation image. In other words, the colony identification section only needs to be able to identify microorganisms forming colonies based on two-dimensional images obtained by irradiating the colonies with visible light and non-visible light. The identification may be performed by analyzing the visible light irradiation image and the non-visible light irradiation image.

Therefore, a configuration in which colonies are identified by processing other than machine learning may be used. For example, when the shape, color, size, pattern, contrast, features of constituent parts, etc. of a colony in each of the visible light irradiation image and the non-visible light irradiation image are acquired, and the features are the features of a specific microorganism, Examples include a configuration that can be identified as a colony formed by the specific microorganism.

In such a configuration, the feature may be defined in advance, and may be defined as an image pattern or as a feature amount. In any case, when the patterns and feature values obtained from the visible light irradiation image and the non-visible light irradiation image match or are similar to the pattern and feature values of the predefined specific microorganism colony. In addition, it can be determined that a colony is formed by the specific microorganism.

For identification, it is sufficient to acquire information about the type of microorganism, and in addition to identifying a specific microorganism, it may be possible to identify the probability of each of a plurality of microorganisms forming a colony. Furthermore, it may be configured to presume that it is not a specific microorganism.
m.

In the above-described embodiments, the camera that detects light in a wavelength range different from the wavelength range of light irradiated to the colonies is a visible light camera that detects visible light that is different from the wavelength range of non-visible light irradiated to the colonies. However, it is not limited to this configuration. For example, a visible light irradiation image may be captured by a non-visible light camera that detects non-visible light different from the wavelength range of the visible light irradiated to the colonies.

In the above-described embodiment, a configuration in which machine learning is performed by YOLO is employed, but of course, various other machine learning techniques can be employed. For example, in a model trained by YOLO, both the identification of colonies and the identification of microorganisms can be performed, but these may be performed separately. Specifically, colonies may be selected from visible light irradiation images and non-visible light irradiation images in which a plurality of colonies are photographed, and the type of microorganism may be identified for each of the selected colonies. Of course, in this case, both colony sorting and microorganism type identification may be performed by machine learning, or at least one of them may be performed by machine learning. When selection or identification is performed without relying on machine learning, it is possible to employ a configuration or the like in which selection or identification is automatically performed by analysis of image patterns or feature amounts.

The mode of machine learning is not limited, for example, when machine learning is performed by a neural network, the number of layers and nodes that make up the model, the type of activation function, the type of loss function, the type of gradient descent method, the gradient Type of descent optimization algorithm, presence or absence of mini-batch learning and number of batches, learning rate, initial value, type and presence of overfitting suppression method, presence or absence of convolution layer, size of filter in convolution operation, type of filter, padding , stride type, pooling layer type and presence/absence, fully connected layer presence/absence, recursive structure presence/absence, and other various factors may be appropriately selected to perform machine learning. Of course, learning may be performed by other machine learning, such as deep learning, support vector machine, clustering, reinforcement learning, and the like.

Furthermore, machine learning may be performed in which the structure of the model (eg, number of layers, number of nodes per layer, etc.) is automatically optimized. Further, machine learning may be performed in a variety of devices, such as in a different device than the colony identification system. In this case, for example, a configuration in which machine learning is performed in the server is assumed. In this configuration, for example, teacher data may be collected from a plurality of clients, and machine learning may be performed in the server based on the teacher data.

Furthermore, the teacher data 30a may be prepared by various methods. Since the training data 30a is information in which the identification result is associated with the image of the colony, it is normally generated by photographing the image of the colony. However, colonies are formed by microorganisms, and their characteristics, such as size, cannot be freely controlled. Therefore, if it is statistically difficult to form a colony with a particular characteristic, it may be difficult to form enough colonies with that characteristic for learning.

In colonies of actinomycetes, etc., it was found to be difficult to form a large number of large and small colonies. Therefore, an image of a colony having a size equal to or larger than the upper limit standard may be generated from an easily obtained image. Also, an image of a colony having a size equal to or smaller than the lower limit standard may be generated from an easily obtained image.

An image that can be easily obtained is assumed to be an image whose size is within the range from the lower limit reference to the upper limit reference, for example. Moreover, when generating an image, the image may be enlarged or reduced. Furthermore, the image generated by enlargement, reduction, etc. may be part of the image, and the rest may be the actual colony image. Furthermore, the image to be generated may be one or both of the visible light irradiation image and the non-visible light irradiation image.

According to the above configuration, it is possible to easily increase the number of images having characteristics that are difficult to be formed by microorganisms, and to easily prepare teacher data. Of course, when an image is generated, various processes other than changing the size may be performed. For example, an image may be generated by performing processing such as rotation and inversion. Furthermore, an image having a size that falls between the upper limit criterion and the lower limit criterion may be generated by padding. Note that the upper limit reference and the lower limit reference should be determined in advance. The upper limit standard is defined in advance as a size that will cause an image size greater than or equal to the upper limit standard to be insufficient, and the lower limit standard will be defined in advance as a size that will cause an image size equal to or smaller than the lower limit standard to become insufficient. .

Furthermore, as in the present invention, the technique of identifying microorganisms forming colonies based on images irradiated with visible light and images irradiated with invisible light can also be applied as a program or method. Moreover, the system, program, and method described above can be implemented as a single device or implemented by a plurality of devices, and include various modes. For example, it is possible to provide a microscope or the like equipped with the means described above. In addition, it can be changed as appropriate, such as a part of which is software and a part of which is hardware. Furthermore, the invention is established as a recording medium for a program for controlling the system. Of course, the recording medium for the software may be a magnetic recording medium, a semiconductor memory, or any other recording medium that will be developed in the future.

DESCRIPTION OF SYMBOLS 10... Colony identification system 20... Control part 21... Colony identification program 21a... Visible light irradiation image acquisition part 21b... Non-visible light irradiation image acquisition part 21c... Colony identification part 22... Machine learning program 22a... Machine learning unit 30 Storage medium 30a Teacher data 30b Learned model 41 Visible light camera 41a Visible light source 42 UV camera 42a UV light source 43 Display unit 44 Output hole

Claims (11)

a visible light irradiation image acquisition unit that acquires a visible light irradiation image, which is an image of a colony of microorganisms formed in a medium that is photographed while the colony is irradiated with visible light;
an invisible light irradiation image acquisition unit that acquires an invisible light irradiation image that is an image of the colony photographed while the colony is irradiated with ultraviolet light that is invisible light;
a colony identification unit that identifies the microorganisms forming the colony based on the visible light irradiation image and the invisible light irradiation image;
with
The colony identification unit is
A model for which machine learning has been performed based on training data that associates the visible light irradiation image and the invisible light irradiation image of the colony with the types of the microorganisms forming the colony, is identified as an identification target. input the visible light irradiation image and the non-visible light irradiation image obtained by photographing the plurality of colonies, and calculate the probability that the microorganism forms each of the plurality of colonies to be identified. Simultaneously output for each type,
Colony identification system. The microorganisms include actinomycetes,
The medium is capable of culturing the actinomycete,
Colony identification system according to claim 1. The visible light irradiation image is formed by reflected light of the visible light reflected by the colony,
The invisible light irradiation image is formed by reflected light of the invisible light reflected by the colony,
A colony identification system according to claim 1 or claim 2. The visible light irradiation image is formed by transmitted light in which the visible light is transmitted through the colony,
The invisible light irradiation image is formed by transmitted light in which the invisible light is transmitted through the colony.
A colony identification system according to any one of claims 1 to 3. At least one of the visible light irradiation image and the non-visible light irradiation image,
Including an image taken with a camera that detects light in a wavelength range different from the wavelength range of light irradiated to the colony,
A colony identification system according to any one of claims 1 to 4. The model is
Simultaneously outputting the positions where the plurality of colonies to be identified are formed and the types of the microorganisms forming each colony;
A colony identification system according to any one of claims 1 to 5. In the teacher data,
At least part of the visible light irradiation image and the invisible light irradiation image of the colony having a size equal to or larger than the upper limit standard and the visible light irradiation image and the invisible light irradiation image of the colony having a size equal to or less than the lower limit standard includes an image generated based on an image of the colony that is less than the upper criterion and greater than the lower criterion.
Colony identification system according to claim 6. The model is
The 3-channel visible light irradiation image captured by the camera that detects the visible light, the 3-channel invisible light irradiation image captured by the camera that detects the visible light, and the invisible light are detected. Each of the 1-channel invisible light irradiation image captured by the camera is input to a separate neural network and processed in parallel, and then the obtained results are synthesized by the neural network,
A neural network that outputs the probability of forming the colony for each type of the microorganism,
A colony identification system according to claim 6 or claim 7 . a visible light camera that captures the visible light irradiation image;
an invisible light camera that captures the invisible light irradiation image;
A transparent plate placed in the field of view of the visible light camera and the invisible light camera and on which a petri dish on which the culture medium containing the colony is formed is placed;
a reflector disposed on the opposite side of the transparent plate from the visible light camera and the non-visible light camera;
further comprising a visible light source that outputs the visible light and an invisible light source that outputs the invisible light, which are arranged in a direction extending the widest surface of the transparent plate;
A colony identification system according to any one of claims 1 to 8. A visible light irradiation image acquisition step of acquiring a visible light irradiation image, which is an image of a colony of microorganisms formed in a medium that is photographed while the colony is irradiated with visible light;
an invisible light irradiation image acquisition step of acquiring an invisible light irradiation image, which is an image of the colony photographed while the colony is irradiated with ultraviolet light, which is invisible light;
a colony identification step of identifying the microorganisms forming the colony based on the visible light irradiation image and the non-visible light irradiation image;
including
In the colony identification step,
A model for which machine learning has been performed based on training data that associates the visible light irradiation image and the invisible light irradiation image of the colony with the types of the microorganisms forming the colony, is identified as an identification target. input the visible light irradiation image and the non-visible light irradiation image obtained by photographing the plurality of colonies, and calculate the probability that the microorganism forms each of the plurality of colonies to be identified. Simultaneously output for each type,
colony identification method. the computer,
a visible light irradiation image acquisition unit that acquires a visible light irradiation image, which is an image of a colony of microorganisms formed in a medium that is photographed while the colony is irradiated with visible light;
an invisible light irradiation image acquisition unit that acquires an invisible light irradiation image that is an image of the colony photographed while the colony is irradiated with ultraviolet light that is invisible light;
a colony identification unit that identifies the microorganisms forming the colony based on the visible light irradiation image and the invisible light irradiation image;
function as
The colony identification unit is
A model for which machine learning has been performed based on training data that associates the visible light irradiation image and the invisible light irradiation image of the colony with the types of the microorganisms forming the colony, is identified as an identification target. input the visible light irradiation image and the non-visible light irradiation image obtained by photographing the plurality of colonies, and calculate the probability that the microorganism forms each of the plurality of colonies to be identified. Make the computer function to output each kind at the same time,
Colony identification program.

Images