JP7391285B2 - Program, information processing device, information processing method, and model generation method - Google Patents

Description

The present invention relates to a program, an information processing device, an information processing method, and a model generation method.

There is a method of detecting bacteria contained in a specimen from an image of the specimen using image processing technology. For example, Patent Document 1 discloses an image processing method for detecting the position of a cell from an image of a specimen using a learning model constructed by deep learning.

JP2019-211468A

However, the invention according to Patent Document 1 may also detect contaminants other than bacteria.

One aspect of the present invention is to provide a program or the like that can suitably detect microorganisms or particulates contained in a specimen.

The program according to one aspect acquires a sample image of a sample that can detect fluorescence, and adds coordinate data of each of microorganisms or particulates and contaminants included in the sample image to the training sample image. The acquired specimen image is inputted into a trained model using training data, and at least microorganisms or particulates are detected. Detect particles, and display the microorganisms or particles commonly detected in the model, the rule base, or another model, and the microorganisms or particles detected in either of them in objects with different display modes. a list showing the number of microorganisms or particulates detected by the model, the number of detections by the rule base or other model, and the number of commonly detected microorganisms or particulates; Have the computer perform the output processing.

In one aspect, microorganisms or particulates contained in a specimen can be suitably detected.

FIG. 2 is an explanatory diagram showing a configuration example of a detection system. FIG. 2 is a block diagram showing a configuration example of a server. FIG. 2 is a block diagram showing an example of the configuration of a terminal. FIG. 2 is a block diagram showing a configuration example of a detection device. FIG. 2 is an explanatory diagram showing an overview of Embodiment 1. FIG. It is an explanatory diagram showing an example of a display screen of a terminal. FIG. 2 is an explanatory diagram showing an example of a document file. 3 is a flowchart showing the procedure of a detection model generation process. It is a flowchart which shows the procedure of a bacteria detection process. 7 is a flowchart showing the procedure of bacteria detection processing according to Embodiment 2. FIG. 3 is a flowchart illustrating a procedure for updating a detection model. FIG. 7 is an explanatory diagram showing an overview of Embodiment 3; FIG. 7 is an explanatory diagram showing an example of a display screen according to Embodiment 3; 12 is a flowchart showing the procedure of bacteria detection processing according to Embodiment 3. FIG. 7 is an explanatory diagram showing an overview of Embodiment 4; FIG. 7 is an explanatory diagram showing an example of a display screen according to Embodiment 4; 12 is a flowchart showing the procedure of bacteria detection processing according to Embodiment 4. FIG. 7 is an explanatory diagram showing an example of a display screen according to Embodiment 5; It is a flowchart which shows the procedure of labeling processing. FIG. 7 is an explanatory diagram showing a document file according to Embodiment 6; 12 is a flowchart showing the procedure of bacteria detection processing according to Embodiment 6. FIG. 7 is an explanatory diagram showing an overview of Embodiment 7; 12 is a flowchart showing the procedure of bacteria detection processing according to Embodiment 7.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on drawings showing embodiments thereof.
(Embodiment 1)
FIG. 1 is an explanatory diagram showing an example of the configuration of a detection system. In this embodiment, a detection system will be described that detects bacteria from a sample image obtained by capturing an image of a sample for bacterial testing of samples such as foods and beverages. The detection system includes an information processing device 1, a terminal 2, a detection device 3, and an administrator terminal 4. The information processing device 1 and the terminal 2 are connected to a network N such as the Internet.

In this embodiment, the test target is bacteria, but the test target is not limited to this. For example, microorganisms other than bacteria such as yeast, mold, and algae, or fine particles such as fluorescent beads may be tested. That is, it is sufficient that microorganisms and/or particulates can be detected, and the test target is not limited to bacteria.

The information processing device 1 is an information processing device capable of various information processing and transmission/reception of information, and is, for example, a server computer, a personal computer, or the like. In this embodiment, the information processing device 1 is assumed to be a server computer, and will be read as server 1 below for brevity. The server 1 performs machine learning to learn training data that is assigned correct coordinate data (labels) for bacteria and contaminants on the training sample images, and uses the detection model 50 (see FIG. 5) to be described later. generate. The detection model 50 is a machine learning model that receives a specimen image as input and detects at least bacteria among bacteria and impurities contained in the specimen image. The server 1 generates a detection model 50 that can distinguish between bacteria and contaminants by learning the coordinate data of not only bacteria but also contaminants other than bacteria.

The terminal 2 is an information processing terminal of a user who uses this system, and is, for example, a personal computer, a tablet terminal, a smartphone, or the like. In this embodiment, the data of the detection model 50 generated by the server 1 is installed in the user's terminal 2, and the terminal 2 inputs a sample image obtained by capturing the sample into the detection model 50 to detect bacteria.

The detection device 3 is an inspection device for detecting bacteria contained in a specimen, and is an imaging device that images the specimen. In this embodiment, a fluorescent staining method is used as a method for detecting bacteria, and the detection device 3 images a specimen stained with a fluorescent stain. The detection device 3 includes a base that holds a measurement filter (for example, a membrane filter) from which a specimen stained with a staining agent has been collected, an XY stage that slides the base in the X-axis direction and the Y-axis direction, and an excitation light that applies excitation light to the specimen. An excitation light source for irradiation, an imaging camera, etc. are provided inside the apparatus, and images of the specimen are taken while finely adjusting the position of the measurement filter in the X-axis direction and the Y-axis direction. As the detection device 3 according to the present embodiment, for example, the detection device described in International Publication No. 2017/115768 can be used. The terminal 2 inputs the sample image captured by the detection device 3 into the detection model 50 and detects bacteria.

Note that the detection device 3 described above is an example, and the detection device 3 only needs to be capable of capturing a sample image. Further, the configuration that uses the detection device 3 for imaging the specimen is not essential; for example, the terminal 2 may use an image of the specimen manually taken as the specimen image. Further, the fluorescent staining method is an example of a bacteria detection method, and the detection method is not limited to the fluorescent staining method as long as bacteria can be detected from an image taken of a specimen.

Furthermore, in this embodiment, the specimen is imaged after being stained, but if the object to be detected (microorganisms or particulates) contains an autofluorescent substance, such as algae, the specimen may not be stained. . That is, the specimen image may be an image of a specimen in which microorganisms or fine particles can be detected by fluorescence, and staining with a fluorescent stain is not essential.

The administrator terminal 4 is an information processing terminal of an administrator who manages this system, and is, for example, a personal computer. The server 1 sends training data, in which the administrator of this system (a staff member who has the skills to identify bacteria and contaminants contained in sample images) to the training sample images with correct labels, to the administrator terminal 4. Obtain from and perform learning.

FIG. 2 is a block diagram showing an example of the configuration of the server 1. As shown in FIG. The server 1 includes a control section 11 , a main storage section 12 , a communication section 13 , and an auxiliary storage section 14 .
The control unit 11 is an arithmetic processing device such as one or more CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), etc., and reads out the program P1 stored in the auxiliary storage unit 14. By executing the command, various information processing, control processing, etc. are performed. The main memory unit 12 is a temporary storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores data necessary for the control unit 11 to perform arithmetic processing. Remember. The communication unit 13 is a communication module for performing processing related to communication, and sends and receives information to and from the outside.

The auxiliary storage unit 14 is a nonvolatile storage area such as a large capacity memory or a hard disk, and stores the program P1 and other data necessary for the control unit 11 to execute processing. Further, the auxiliary storage unit 14 stores a detection model 50. The detection model 50 is a machine learning model that has already learned the training data as described above, and is a trained model that detects bacteria included in the sample image by inputting a sample image obtained by capturing a sample stained with a staining agent. . The detection model 50 is assumed to be used as a software module that forms part of artificial intelligence software.

Note that the auxiliary storage unit 14 may be an external storage device connected to the server 1. Further, the server 1 may be a multicomputer consisting of a plurality of computers, or may be a virtual machine virtually constructed by software.

Further, in this embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input section that receives operation input, a display section that displays images, and the like. The server 1 also includes a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk)-ROM or a DVD (Digital Versatile Disc)-ROM, and is configured to read and execute the program P1 from the portable storage medium 1a. You can also do it. Alternatively, the server 1 may read the program P1 from the semiconductor memory 1b.

FIG. 3 is a block diagram showing an example of the configuration of the terminal 2. As shown in FIG. It includes a control section 21, a main storage section 22, a communication section 23, a display section 24, an input section 25, and an auxiliary storage section 26.
The control unit 21 is an arithmetic processing device such as one or more CPUs or MPUs, and performs various information processing, control processing, etc. by reading and executing the program P2 stored in the auxiliary storage unit 26. The main storage unit 22 is a temporary storage area such as a RAM, and temporarily stores data necessary for the control unit 21 to perform arithmetic processing. The communication unit 23 is a communication module for performing communication-related processing, and sends and receives information to and from the outside.

The display unit 24 is a display screen such as a liquid crystal display, and displays images. The input unit 25 is an operation interface such as a keyboard and a mouse, and receives operation input from the user. The auxiliary storage unit 26 is a nonvolatile storage area such as a hard disk or a large capacity memory, and stores the program P2 and other data necessary for the control unit 21 to execute processing. Further, the auxiliary storage unit 26 stores a detection model 50.

Note that the terminal 2 may include a reading unit that reads a portable storage medium 2a such as a CD-ROM, and may read and execute the program P2 from the portable storage medium 2a. Alternatively, the terminal 2 may read the program P2 from the semiconductor memory 2b.

FIG. 4 is a block diagram showing an example of the configuration of the detection device 3. As shown in FIG. The detection device 3 includes a control section 31 , an imaging section 32 , an XY-axis motor driver 33 , an XY-axis motor 34 , an LD (Laser Diode) driver 35 , and an LD light source 36 .
The control unit 31 is an arithmetic processing device such as a CPU, and performs various information processing to control the detection device 3. The imaging unit 32 is a camera having an imaging element such as a CCD (Charge Coupled Device) sensor, and images the specimen collected by the measurement filter. The XY-axis motor driver 33 is a driver that controls the operation of the XY-axis motor 34 that drives the XY stage, and adjusts the position of the measurement filter placed on the base on the XY stage. The LD driver 35 is a driver that adjusts the amount of excitation light emitted from the LD light source 36, which is an excitation light source, and irradiates the sample with the excitation light.

FIG. 5 is an explanatory diagram showing an overview of the first embodiment. FIG. 5 conceptually illustrates how bacteria included in a specimen image are detected when the specimen image photographed by the detection device 3 is input to the detection model 50. An overview of this embodiment will be explained based on FIG. 5.

The terminal 2 acquires a specimen image from the detection device 3. The detection device 3 is an inspection device for detecting bacteria as described above, and is a device for imaging a specimen stained with a fluorescent stain. The detection device 3 irradiates excitation light onto a measurement filter that has collected the stained specimen, and captures a specimen image (fluorescence image). Although a detailed explanation will be omitted, the detection device 3 divides the area where the sample was collected on the measurement filter into a plurality of grid-like sections along the X-axis direction and the Y-axis direction, as shown in the upper left of FIG. . Then, the detection device 3 captures a specimen image corresponding to each section while sliding the measurement filter placed on the base on the XY stage in the X-axis direction and the Y-axis direction. The detection device 3 transmits the captured specimen image to the terminal 2.

Note that although the specimen image captured by the detection device 3 is a black and white image, the specimen image may be an RGB image, and its image format is not particularly limited.

The terminal 2 inputs the specimen image transmitted from the detection device 3 into the detection model 50. The detection model 50 is a machine learning model generated by learning training data, and is, for example, a neural network constructed by deep learning. Specifically, the detection model 50 is a CNN (Convolution Neural Network) such as R-CNN (Region CNN), YOLO (You Look Only Once), or SSD (Single Shot Detector).

In this embodiment, the detection model 50 will be described as a CNN, but it may be a neural network other than CNN, a decision tree, a random forest, or other machine learning model.

The detection model 50 includes an input layer that receives input of a specimen image, an intermediate layer that extracts features from the specimen image, and an output layer that outputs bacterial detection results (coordinate data) based on the extracted features. The input layer receives input of specimen images and passes the data to the intermediate layer. The intermediate layer has a configuration in which convolution layers and pooling layers are alternately connected, compresses the pixel value of each pixel of the specimen image, extracts a feature amount, and passes it to the output layer. The output layer outputs coordinate data in the specimen image in which bacteria are estimated to exist, based on the features extracted in the intermediate layer.

The server 1 performs learning on the training sample image using training data to which correct coordinate data (labels) of bacteria are added, and generates the detection model 50 . Here, the server 1 performs learning using training data to which not only bacteria coordinate data but also contaminant coordinate data is added.

Contaminants refer to substances other than bacteria that are not targeted for detection. If the specimen contains foreign matter (for example, dust such as fibers), the foreign matter may also be stained by the staining agent, and the foreign matter may also be observed as a luminescent spot in the specimen image. Furthermore, if the sample contains an autofluorescent substance (for example, flour contained in food), the substance may be observed as a luminescent spot and become a contaminant in detecting bacteria. Skilled workers can distinguish these contaminants from bacteria, but the manual labor requires longer testing times and limits the number of specimens that can be processed.

Therefore, in this embodiment, the detection model 50 is made to learn the characteristics of not only bacteria but also contaminants, and the detection model 50 that can discriminate between bacteria and contaminants is constructed. That is, the server 1 performs learning on the training detection images using training data to which coordinate data of bacteria and contaminants are added, and generates the detection model 50.

For example, the training data is created by an administrator (a staff member with specialized knowledge about bacterial testing) on the administrator terminal 4 and provided to the server 1 . The server 1 inputs training sample images to the detection model 50 and obtains coordinate data on which bacteria are estimated to exist as output. The server 1 compares the acquired coordinate data with the correct coordinate data, and adjusts parameters such as weights between neurons so that the two approximate each other. For example, the server 1 performs learning on all sample images given for training, and finally generates a detection model 50 with optimized parameters.

Note that the detection model 50 according to the present embodiment outputs only coordinate data of bacteria as a detection result, but may output coordinate data of not only bacteria but also contaminants. That is, the detection model 50 only needs to be able to detect at least bacteria among bacteria and contaminants, and may also detect contaminants.

The server 1 may have one detection model 50 learn about all types of bacteria (and contaminants) so that one detection model 50 can detect all types of bacteria. It is preferable to learn and prepare a plurality of detection models 50, 50, 50, etc. according to the type of bacteria. That is, the server 1 stores training data in which bacteria A and contaminant labels are attached to training sample images, training data in which bacteria B and contaminant labels are attached to training sample images, and training data in which bacteria B and contaminant labels are attached to training sample images. Training data in which images are labeled with bacteria C and contaminants are acquired, and based on each training data, a detection model 50 for detecting bacteria A, a detection model 50 for detecting bacteria B, and a detection model 50 for detecting bacteria C are acquired. A detection model 50... is generated. The user selects the detection model 50 among these detection models 50, 50, 50, . . . according to the test purpose (type of bacteria to be detected) and uses it for bacteria detection.

In the terminal 2, data of a detection model 50 generated by the server 1 is installed. The terminal 2 inputs the specimen image acquired from the detection device 3 into the detection model 50, and detects bacteria included in the specimen image. On the right side of FIG. 5, a bounding box (rectangular frame) illustrates how bacteria are detected. As shown in FIG. 5, the detection model 50 detects bacteria indicated by black circles, while identifying substances indicated by hatched squares as contaminants and excludes them from the detection results.

FIG. 6 is an explanatory diagram showing an example of a display screen of the terminal 2. As shown in FIG. FIG. 6 illustrates an operation screen displayed by the terminal 2 when performing an inspection using the detection device 3. Based on FIG. 6, the operating procedure performed by the user during the examination will be described.

The screen includes an operation field 61, a tab 62, and an output button 63. The operation column 61 is a display column that displays the current state of the detection device 3 and also displays icon buttons for controlling the operation of the detection device 3. By operating the icon buttons, the user controls the operation of the XY stage, the start of photographing the specimen image, and the like.

The tab 62 is an operation tab for switching screen display. When the "detection conditions" tab 62 is active, the terminal 2 switches the display to a screen (not shown) for setting detection conditions, and accepts settings for various detection conditions. For example, the terminal 2 sets the detection conditions as the section on the measurement filter as the imaging range, the sample volume (ml), the sample number, the type of test reagent (staining agent) used, and the image capturing conditions (brightness correction, mask correction, etc.). ) etc. are accepted. The terminal 2 also accepts a selection input of the type of detection model 50 (type of bacteria) used for bacteria detection as a detection condition.

After completing the setting of the detection conditions, the terminal 2 causes the detection device 3 to capture a sample image of each section on the measurement filter in response to an operation input to the icon button in the operation field 61. Then, the terminal 2 inputs the captured sample image of each section to the detection model 50 selected above, and detects bacteria.

When the "detection results" tab 62 is active, the terminal 2 displays the bacteria detection results. Specifically, as shown in FIG. 6, the terminal 2 displays a sample image (second sample image) with bounding boxes (objects) attached at positions corresponding to detected bacteria, and the number of detected bacteria. Display. The terminal 2 sequentially displays the sample images of each section on the measurement filter according to the user's operation. In addition, the terminal 2 displays a list of the number of bacteria detected for each section (detection location) on the measurement filter in a table 621, and also displays the number of bacteria detected in the section related to the currently displayed sample image at the top of the table 621. do.

In addition, on the screen of FIG. 6, the terminal 2 may display the degree of certainty indicating the probability that the detection model 50 has detected the bacteria at each detection point, with respect to the bacteria at each detection point indicated by the bounding box. The confidence level is a probability value representing the probability that the detection model 50 detects each bacterium based on the feature amount of the specimen image, and is output from the detection model 50 together with the coordinate data of each detection point. For example, when the cursor is placed on a bounding box in response to a user's operation, the terminal 2 displays the confidence level corresponding to the bounding box (detection point). Thereby, it is possible to present to the user how likely the detection result of the detection model 50 is.

The output button 63 is an operation button for outputting a storage file (hereinafter referred to as a "save file") that summarizes the bacterial detection results. When the terminal 2 receives an operation input to the output button 63, the terminal 2 outputs (exports) the saved file. Specifically, the terminal 2 stores an image file of the specimen image (a file in which the raw specimen image acquired from the detection device 3 is associated with the specimen image with a bounding box (second specimen image)), and a table. A text file describing the number of detected bacteria displayed in step 621 is output.

Furthermore, the terminal 2 generates and outputs a document file in which the bacterial detection results are described in a report format as one of the saved files. FIG. 7 is an explanatory diagram showing an example of a document file. The terminal 2 generates the document file illustrated in FIG. For example, the terminal 2 generates a document file that describes detection conditions such as the test reagent used and the exposure time of excitation light, and also describes the number of detections per unit amount (ml) as a detection result. The terminal 2 calculates the number of bacteria detected per unit amount based on the sample amount set as the detection condition and the number of bacteria detected from the sample image, writes it in a document file, and writes it to the document file. Output along with the file.

As described above, by using the detection model 50 that has learned the characteristics of not only bacteria but also contaminants, the terminal 2 can exclude contaminants from detection targets and appropriately detect bacteria.

FIG. 8 is a flowchart showing the procedure for generating the detection model 50. Based on FIG. 8, the processing contents when generating the detection model 50 by machine learning will be explained.

The server 1 acquires training data for generating the detection model 50 from the administrator terminal 4 (step S11). The training data is data in which correct coordinate data (labels) of bacteria and contaminants are added to the training sample image.

Based on the training data, the server 1 generates a detection model 50 that detects at least bacteria when a specimen image is input (step S12). For example, the server 1 generates a neural network such as CNN as the detection model 50. The server 1 inputs a training specimen image to the detection model 50 and obtains as output coordinate data in the specimen image in which bacteria are estimated to exist. The server 1 compares the acquired bacterial coordinate data with the correct coordinate data, optimizes the weights between neurons, etc. so that the two approximate each other, and generates the detection model 50. Server 1 ends the series of processing.

FIG. 9 is a flowchart showing the procedure of bacteria detection processing. Based on FIG. 9, the processing details when detecting bacteria from a sample image will be described.
The terminal 2 accepts setting input of bacteria detection conditions (step S31). For example, the terminal 2 accepts setting inputs such as the division on the measurement filter as the imaging range, the amount of specimen, etc., and also accepts the selection input of the type of detection model 50 (type of bacteria) to be used.

The terminal 2 acquires a sample image of the sample from the detection device 3 (step S32). The terminal 2 inputs the acquired specimen image into the detection model 50 selected above to detect bacteria (step S33). The terminal 2 totals the number of detected bacteria (step S34). The terminal 2 displays the bacteria detection results (step S35). Specifically, the terminal 2 displays a specimen image (second specimen image) with a bounding box (object) attached at a position corresponding to the detected bacteria, and also displays the number of detections totaled in step S34.

The terminal 2 determines whether or not to output the saved file based on the operation input from the user (step S36). If it is determined that the saved file is to be output (S36: YES), the terminal 2 outputs the saved file (step S37). For example, the terminal 2 generates an image file of the specimen image, etc., as well as a document file in which the detection results including the number of bacteria detected are described in a report format, and outputs it as a save file. After executing the process in step S37, or in the case of NO in step S36, the terminal 2 ends the series of processes.

As described above, according to the first embodiment, bacteria contained in a specimen can be suitably detected.

Further, according to the first embodiment, the number of detected bacteria can be totaled and presented to the user as a detection result together with a specimen image with a bounding box.

Further, according to the first embodiment, by preparing a plurality of detection models 50 that have been trained with training data according to the type of bacteria, it is possible to suitably detect bacteria to be detected.

(Embodiment 2)
In the first embodiment, a case has been described in which the local terminal 2 uses the detection model 50 to perform bacteria detection processing. In this embodiment, a configuration in which the server 1 on the cloud performs bacteria detection processing will be described. Note that the same reference numerals are given to the same content as in the first embodiment, and the explanation thereof will be omitted.

First, an overview of this embodiment will be explained. In this embodiment, the server 1 receives a request from the terminal 2 and performs bacteria detection processing using the detection model 50. That is, the server 1 acquires the sample image captured by the detection device 3 from the terminal 2, inputs it to the detection model 50, and detects bacteria. The server 1 then outputs the detection results to the terminal 2 for display. Since the display screen and the like are the same as those in Embodiment 1, illustrations are omitted in this embodiment.

Here, it is preferable that the server 1 re-learns (updates) the detection model 50 based on the specimen image acquired from the terminal 2 using a cloud environment. For example, the server 1 stores sample images acquired from the terminal 2 and periodically outputs them to the administrator terminal 4. The administrator terminal 4 inputs an operation to add correct coordinate data (labels) of bacteria and contaminants to the specimen image output from the server 1 (for example, adds a bounding box as in Embodiment 5 described later). or deletion operation input) from the administrator, and outputs it to the server 1 as training data for relearning.

The server 1 updates the detection model 50 based on the training data for relearning. That is, the server 1 uses the sample image for relearning (the sample image acquired from the terminal 2) as the detection model 50 used for bacteria detection (i.e., the detected bacteria among the plurality of detection models 50, 50, 50...). The parameters of the detection model 50 are optimized so that the detection result approximates the correct coordinate data. Thereby, the detection model 50 is updated sequentially, and detection accuracy can be improved.

Note that in this embodiment, the server 1 executes both the bacteria detection process (inference process using the detection model 50) and the relearning process, but the server 1 may only perform the relearning process. .

Further, the server 1 may perform relearning for each type of specimen and construct detection models 50, 50, 50, etc. suitable for each type of specimen. For example, when detecting bacteria by acquiring sample images from users such as food companies, beverage manufacturers, and waterworks bureaus, the server 1 can store images for relearning for each type of sample such as food, beverages, and tap water. Training data is provided to separate detection models 50 to update the detection models 50, 50, 50, . . . for each type of analyte. Thereby, through operation of this system, it is possible to construct a detection model 50 suitable for detecting bacteria in individual specimens.

FIG. 10 is a flowchart showing the procedure of bacteria detection processing according to the second embodiment. After acquiring the specimen image from the detection device 3 (step S32), the terminal 2 executes the following process.
The terminal 2 transmits the acquired specimen image to the server 1 (step S201). When the sample image is acquired from the terminal 2, the server 1 inputs the sample image into the detection model 50 to detect bacteria (step S202). The server 1 transmits the detection result (bacteria coordinate data) to the terminal 2 (step S203). The server 1 saves (memorizes) the sample image acquired from the terminal 2 (step S204), and ends the series of processing.

FIG. 11 is a flowchart showing the procedure for updating the detection model 50. Based on FIG. 11, the processing contents when updating the detection model 50 by performing relearning will be described.
The server 1 outputs the stored sample image to the administrator terminal 4 (step S221). The administrator terminal 4 receives an operation input for adding correct coordinate data (labels) of bacteria and contaminants to the specimen image (step S222). The administrator terminal 4 transmits the sample image to which the correct coordinate data has been added to the server 1 as training data for relearning (step S223). The server 1 updates the detection model 50 based on the training data for relearning (step S224). Server 1 ends the series of processing.

As described above, according to the second embodiment, the server 1 on the cloud instead of the local terminal 2 may execute the bacteria detection process. In addition, by performing relearning based on the sample image on which bacteria have been detected, the accuracy of bacteria detection can be improved.

(Embodiment 3)
In the first embodiment, a plurality of detection models 50, 50, 50, etc. are prepared according to the type of bacteria to be detected, and one of the detection models 50 is selected and used for bacteria detection. In this embodiment, a mode in which a plurality of detection models 50, 50, 50, . . . are used together will be described.

FIG. 12 is an explanatory diagram showing an overview of the third embodiment. FIG. 12 conceptually illustrates how each type of bacteria is individually detected by inputting the same specimen image to a plurality of detection models 50, 50, 50, . . . .

As described in the first embodiment, the server 1 generates a plurality of detection models 50, 50, 50, . . . according to the types of bacteria, based on training data corresponding to bacteria A, B, C, . . . . In this embodiment, the terminal 2 inputs the specimen image acquired from the detection device 3 to a plurality of detection models 50, 50, 50, . . . , respectively. Then, the terminal 2 obtains coordinate data of bacteria A, B, C, . . . from each detection model 50 as output.

The terminal 2 combines and displays the detection results of each detection model 50. FIG. 13 is an explanatory diagram showing an example of a display screen according to the third embodiment. In the present embodiment, when displaying a specimen image with a bounding box attached to the position of the bacteria, the terminal 2 displays the specimen image (second specimen image) in which the bounding box is displayed in a different manner depending on the type of bacteria. Display. For example, the terminal 2 changes the display color of the bounding box. Note that in FIG. 13, for convenience of illustration, different display colors are expressed by line types.

Furthermore, the terminal 2 displays a list of the number of each type of bacteria detected for each section (detection location) in a table 621. Furthermore, the terminal 2 displays the number of detected bacteria of each type in the section related to the currently displayed specimen image at the top of the table 621.

FIG. 14 is a flowchart showing the procedure of bacteria detection processing according to the third embodiment. After acquiring the specimen image from the detection device 3 (step S32), the terminal 2 executes the following process.
The terminal 2 inputs the specimen image acquired in step S32 into a plurality of detection models 50, 50, 50, etc. that have already learned training data according to the type of bacteria, and detects a plurality of types of bacteria (step S301). ).

The terminal 2 totals the number of detected bacteria of each type (step S302). The terminal 2 then displays the detection results of each type of bacteria (step S303). Specifically, the terminal 2 displays a specimen image (second specimen image) with a bounding box in a different display mode depending on the type of bacteria, and also displays the number of detections for each type of bacteria. The terminal 2 moves the process to step S36.

As described above, according to the third embodiment, multiple types of bacteria can be detected simultaneously.

Note that in this embodiment as well, the server 1 may perform relearning of the detection model 50. For example, the server 1 receives an input specifying the type of bacteria, and then receives an operation input for providing correct coordinate data of the specified bacteria, thereby acquiring training data for relearning. The server 1 updates the detection model 50 corresponding to the designated type of bacteria based on the original specimen image and the newly assigned correct coordinate data.

(Embodiment 4)
In this embodiment, in addition to detecting bacteria using the detection model 50, a mode will be described in which bacteria are detected by rule-based image processing and the combination of the detection model 50 and the rule-based detection results is presented to the user. .

FIG. 15 is an explanatory diagram showing an overview of the fourth embodiment. FIG. 15 conceptually illustrates how bacteria are detected from a specimen image using the detection model 50 and bacteria are detected from the specimen image by rule-based image processing (for example, binarization processing).

Similar to the first embodiment, the terminal 2 inputs the sample image acquired from the detection device 3 to the detection model 50 and detects bacteria. The detection results by the detection model 50 are conceptually illustrated in the upper right corner of FIG. Note that, in order to distinguish it from the detection result (rule-based detection result) shown in the lower right of FIG. 15, the detection result by the detection model 50 is shown in a round frame instead of a rectangular bounding box in FIG.

Furthermore, the terminal 2 performs rule-based image processing to detect bacteria from the sample image. For example, the terminal 2 performs a binarization process to binarize the sample image as rule-based image processing to detect bacteria. That is, the terminal 2 compares each pixel value (for example, brightness) in the specimen image with a predetermined threshold, binarizes the specimen image into a region estimated to be bacteria and a region other than bacteria, and determines the area estimated to be bacteria. detect the area where

Note that the binarization process is an example of a rule-based detection method, and other algorithms may be used to detect bacteria from the specimen image.

The lower right of FIG. 15 conceptually illustrates the detection results based on the rule. When the detection result by the detection model 50 and the rule-based detection result are compared in the example of FIG. 15, it can be seen that bacteria that should originally be detected are not detected in the detection result by the detection model 50. Furthermore, the detection results obtained by the detection model 50 indicate that a plurality of bacteria (two) are close to each other, so that a plurality of bacteria are detected as one bacterium. On the other hand, it can be seen that in the rule-based detection results, image areas corresponding to foreign objects are also detected, resulting in a large number of erroneous detection points.

The terminal 2 displays a combination of the above two detection results. FIG. 16 is an explanatory diagram showing an example of a display screen according to the fourth embodiment. The display screen according to this embodiment includes an image tab 622. The image tab 622 is an operation tab for displaying a specimen image by switching the detection model 50 and/or the rule-based detection result. When the "AI Detection" image tab 622 is activated, the terminal 2 displays a specimen image showing the position of the bacteria detected by the detection model 50 with a bounding box (see upper right of FIG. 15). When the image tab 622 for “binarization detection” is active, the terminal 2 displays a specimen image that shows the position of bacteria detected by the binarization process with a bounding box (see bottom right of FIG. 15). .

When the image tab 622 of “Comparison of AI detection and binarization detection” is activated, the terminal 2 detects a sample that shows the detection results of both the detection model 50 and the binarization process, as shown in FIG. Display images. Specifically, the terminal 2 detects a detection point (bacteria) commonly detected by the detection model 50 and the binarization process, and a detection point detected by either the detection model 50 or the binarization process. Change the display mode (for example, display color) of the bounding box.

In addition, terminal 2 displays a list of the number of detections by the detection model 50, the number of detections by binarization processing, and the number of commonly detected bacteria for each section (detection location) on the measurement filter in table 621. do. In addition, the terminal 2 displays the number of detections by the detection model 50, the number of detections by binarization processing, and the number of commonly detected bacteria for the section related to the currently displayed specimen image in the upper part of the table 621. indicate.

FIG. 17 is a flowchart showing the procedure of bacteria detection processing according to the fourth embodiment. After acquiring the specimen image from the detection device 3 (step S32), the terminal 2 executes the subsequent processing.
The terminal 2 inputs the sample image into the detection model 50 to detect bacteria (step S401). Furthermore, the terminal 2 detects bacteria from the sample image through binarization processing (step S402). The terminal 2 totals the number of bacteria detected in steps S401 and S402 (step S403). Specifically, the terminal 2 counts the number of bacteria detected by the detection model 50 and the number of bacteria detected by the binarization process, as well as the number of bacteria detected in common by the detection model 50 and the binarization process. Tally.

The terminal 2 displays the detection results by the detection model 50 and the detection results by the binarization process (step S404). Specifically, the terminal 2 displays different displays for bacteria detected only by the detection model 50 or the binarization process and bacteria detected in common by the detection model 50 and the binarization process. A specimen image (second specimen image) with a bounding box (object) of the aspect is displayed. The terminal 2 also displays the number of bacteria detected by the detection model 50, the number of bacteria detected by the binarization process, and the number of bacteria detected in common by the detection model 50 and the binarization process. The terminal 2 moves the process to step S36.

As described above, according to the fourth embodiment, by combining rule-based image processing, more detection results can be presented to the user.

Note that in this embodiment, rule-based image processing (binarization processing) is used as another detection algorithm combined with the detection model 50, but the present embodiment is not limited to this. For example, the terminal 2 may use another machine learning model different from the detection model 50 as another detection algorithm to be combined with the detection model 50. That is, the terminal 2 inputs the specimen image to the detection model 50 and other models, and obtains detection results from each model. The other model may be a model that has the same network structure as the detection model 50 but has learned different training data, or may be another model that has a different network structure. Even if other models are used in place of the rule-based image processing, the same effects as described above can be achieved.

(Embodiment 5)
In the fourth embodiment, an embodiment in which the detection model 50 and rule-based detection processing are combined has been described. In this embodiment, a mode will be described in which a sample image is labeled for re-learning using the detection model 50 and the rule-based detection results.

FIG. 18 is an explanatory diagram showing an example of a display screen according to the fifth embodiment. In this embodiment, as one of the functions for re-analyzing specimen images that have been previously subjected to bacteria detection processing and saved as saved files, we provide a function that adds correct bacterial coordinate data (labels) to specimen images. provide. An overview of this embodiment will be explained based on FIG. 18.

When the terminal 2 receives an operation input to the "import image" button, the terminal 2 receives an input specifying a specimen image saved as a save file. The terminal 2 then displays the specified sample image on the screen. The screen includes an image tab 181, an add button 182, a delete button 183, and a save button 184.

The image tab 181 is an operation tab for switching the display of specimen images. When the "original image" image tab 181 is activated, the terminal 2 displays the original specimen image (raw image without a bounding box) captured by the detection device 3. When the "AI Detection" image tab 181 is activated, the terminal 2 displays a specimen image showing the position of the bacteria detected by the detection model 50 with a bounding box (see upper right of FIG. 15). When the image tab 181 of “binarization detection” is activated, the terminal 2 displays a specimen image showing the position of bacteria detected by the binarization process with a bounding box (see lower right of FIG. 15). .

When the image tab 181 of “Comparison of AI detection and binarization detection” is activated, the terminal 2 displays a sample image showing the detection results of both the detection model 50 and the binarization process. Specifically, as in the fourth embodiment, the terminal 2 detects a detection point (bacteria) commonly detected by the detection model 50 and the binarization process, and either the detection model 50 or the binarization process. A specimen image with a bounding box of a different display mode is displayed depending on the detection point detected in .

The terminal 2 accepts an operation input to add or delete a detection point (bacteria) displayed as a bounding box on the screen. For example, the terminal 2 receives a designation input specifying a bounding box, and adds or deletes the designated bounding box to the detection point in response to an operation input to the add button 182 or the delete button 183. FIG. 18 illustrates how the bounding box specified by the user is represented by an arrow icon.

As described above, the terminal 2 displays the detection points commonly detected by the detection model 50 and the binarization process, and the detection points detected by either one in different display modes. Thereby, when the user selects a detection point to which a label should be attached, it is possible to suitably assist the user in the selection work.

Note that the terminal 2 may accept an operation input for adding a label (adding a bounding box) not only to bacteria but also to foreign matter. As a result, it is possible to obtain training data in which correct labels are added even for contaminants.

When accepting an operation input to the save button 184, the terminal 2 saves the contents of addition and/or deletion of detection points. Specifically, the terminal 2 adds and/or deletes detection points added and/or deleted by the user as bacterial locations from the detection results of the detection model 50 stored as a storage file, and updates the storage file. do.

The server 1 can relearn the detection model 50 using the saved file in which detection points have been added or deleted as described above. That is, the server 1 updates the detection model 50 by giving the detection points added and/or deleted by the user as a correct label to the detection model 50. The server 1 inputs the sample image in the saved file to the detection model 50 and obtains coordinate data of the bacteria (detection point) as output. The server 1 compares the coordinate data output from the detection model 50 with the coordinate data of the detection point added and/or deleted by the user, and optimizes the parameters of the detection model 50 so that the two approximate each other.

Since the relearning process was also explained in the second embodiment, the flowchart and other detailed explanations of the relearning process will be omitted in this embodiment.

FIG. 19 is a flowchart showing the procedure of labeling processing. Based on FIG. 19, details of processing executed by the terminal 2 according to the fifth embodiment will be described.
The terminal 2 reads the saved file stored in the auxiliary storage unit 26 (step S501). The terminal 2 then displays the specimen image (step S502). Specifically, the terminal 2 detects detection points (bacteria) detected only by either the detection model 50 or the binarization process, and detection points detected in common by the detection model 50 and the binarization process. (bacteria), a specimen image (second specimen image) with a bounding box (object) of a different display mode is displayed.

The terminal 2 accepts an operation input for adding or deleting a detection point indicated by a bounding box to the displayed specimen image (step S503). The terminal 2 saves the image file after adding or deleting the detection points (step S504), and ends the series of processing.

As described above, according to the fifth embodiment, training data for relearning can be suitably created.

(Embodiment 6)
In this embodiment, a mode will be described in which bacteria detection (inspection) is performed multiple times on the same specimen.

FIG. 20 is an explanatory diagram showing a document file according to the sixth embodiment. FIG. 20 illustrates a document file that summarizes the detection results of each test when the same specimen is tested multiple times. An overview of this embodiment will be explained based on FIG. 20.

A case in which the same specimen is tested multiple times is, for example, a case in which HACCP (Hazard Analysis and Critical Control Point) is performed. HACCP refers to a method of managing important processes in the food manufacturing process in order to eliminate or reduce risk factors such as bacteria (food poisoning bacteria) and foreign substance contamination. In this embodiment, a case will be described assuming that, as part of HACCP, an inspection is performed in each of a plurality of processes for manufacturing food (sample).

The terminal 2 acquires specimen images in each of the plurality of steps, inputs the specimen images to the detection model 50, and detects bacteria. The terminal 2 then outputs the bacteria detection results in each step in the form of a document file or the like. Here, when the terminal 2 performs an inspection in one process and outputs the detection result, it outputs the detection result in a form that can be compared with the inspection in the previous process.

For example, terminal 2 refers to the saved file saved during the previous inspection and creates a document that shows the difference between the detection results such as the number of detections from the previous inspection and the current detection result (step 1 above). Generate and output files. For example, when setting the detection conditions, terminal 2 accepts the specified input of the previous saved file, refers to the specified saved file, and calculates the difference between the number of detections during the previous inspection and the current number of detections. and write it in the document file.

FIG. 20 shows an example of a document file in which differences in detection results are described. For example, terminal 2 writes the number of bacteria detected per unit amount detected this time as "6. Current detection result" and calculates the difference from the number of bacteria detected last time (second time in Figure 20). Write in the document file. Furthermore, the terminal 2 writes a list of the number of detections detected during each test before the previous time and the difference from the previous time as "7. Detection results up to the previous time." As a result, when testing the same specimen in multiple steps as in HACCP, the test results in each step can be suitably summarized and reported to the user.

FIG. 21 is a flowchart showing the procedure of bacteria detection processing according to the sixth embodiment.
First, the terminal 2 receives an input of detection condition settings (step S601). For example, the terminal 2 accepts designation input of a past saved file to be compared, that is, a detection result before the previous time, in addition to the section on the measurement filter to be imaged, the amount of specimen, the specimen number, etc. The terminal 2 reads the specified save file (step S602). The terminal 2 acquires the sample image to be examined this time (step S603), and moves the process to step S33.

If it is determined to output a save file (S36: YES), the terminal 2 generates a document file describing the detection results, and outputs it as a save file together with the image file of the specimen image, etc. (Step S604). Here, the terminal 2 refers to the saved file from the previous inspection and generates a document file that describes the detection results in each of the plurality of manufacturing processes. Specifically, as described above, the terminal 2 generates a document file indicating the difference in the number of bacteria detected in each step. Terminal 2 ends the series of processing.

As described above, according to the sixth embodiment, it is possible to test a sample such as a food product according to its manufacturing process, and to suitably present the detection results in each process.

(Embodiment 7)
In this embodiment, a mode will be described in which bacteria are detected using the detection model 50 after enlarging a specimen image.

FIG. 22 is an explanatory diagram showing an overview of the seventh embodiment. FIG. 22 illustrates how a specimen image is enlarged to a predetermined magnification and divided into a plurality of images, and bacteria are detected from each divided image. An overview of this embodiment will be explained based on FIG. 22.

Note that hereinafter, an image obtained by enlarging the specimen image to a predetermined magnification will be referred to as an "enlarged image", and an image obtained by dividing the enlarged image into a plurality of parts will be referred to as a "divided image".

As described in the first embodiment, the terminal 2 uses the detection model 50 to detect bacteria from the specimen image. However, the size of bacteria is very small, several micrometers, so if the lens magnification of the camera used for imaging is small, the bacteria (light spot) observed in the specimen image will also be small. Therefore, if the annotation size in the training data during learning is large, there is a risk that multiple light spots that are close to each other, as shown in the upper right corner of FIG. 15, will be erroneously detected as one bacterium. On the other hand, if the annotation size is made smaller, multiple light spots can be detected individually, but the smaller the annotation size, the less information is used around the light spots for detection, which has a negative impact on identification with foreign objects, etc. There is a risk of giving.

Therefore, in this embodiment, a specimen image is enlarged to a predetermined magnification, and bacteria are detected from the enlarged image. Specifically, the terminal 2 divides the enlarged image into a plurality of divided images according to the enlargement magnification, and sequentially inputs each divided image to the detection model 50, thereby detecting bacteria from each divided image.

FIG. 22 illustrates how the original specimen image is enlarged twice and the specimen image is divided into 2×2=4 divided images. Note that the reason why the divided images are divided into four images is to match the image size during learning. Terminal 2 enlarges the specimen image twice. In this case, the terminal 2 performs image interpolation processing (pixel interpolation using linear interpolation or the like) in order to smooth the image seams. The terminal 2 divides the enlarged image into four equal parts vertically and generates divided images. The terminal 2 inputs each divided image to the detection model 50 and detects bacteria.

Although the enlarged image is divided into a plurality of divided images in this embodiment, the detection model 50 may be used to directly detect bacteria from the enlarged image without dividing the enlarged image.

Further, in the above description, the interpolation is performed by simply enlarging the original specimen image, but for example, the terminal 2 may generate an enlarged image using a machine learning model for generating an enlarged image. In this way, there is no particular limitation on the method of generating the enlarged image.

As described above, the terminal 2 detects bacteria from each segmented image. On the other hand, if bacteria are located near the boundary between two adjacent divided images, the bacteria may be detected redundantly from each of the two divided images, resulting in double counting. Therefore, the terminal 2 specifies, among the bacteria detected from each divided image, the bacteria that are detected redundantly in two adjacent divided images.

Specifically, the terminal 2 identifies bacteria detected from the boundary between two adjacent divided images and from both of the two divided images. Note that the “border portion” refers to an area within a predetermined distance (for example, within several pixels) from the boundary of the divided image (the edge of the divided image) in a direction perpendicular to the boundary line. For example, the terminal 2 identifies bacteria detected from the same boundary portion in the two divided images and located at the same position in the direction parallel to the boundary line, as the bacteria detected overlappingly. Note that "the positions in the direction parallel to the boundary line are the same" refers to a state in which the coordinate values in the direction parallel to the boundary line completely match or are within a predetermined value (for example, within several pixels).

For example, when considering two divided images that are vertically divided into two, terminal 2 detects bacteria detected near the bottom of the upper divided image (border area) and bacteria detected near the bottom edge of the lower divided image (border area). ), it is determined whether the position of each bacterium in the left-right direction is the same. If it is determined that they are the same, the terminal 2 identifies the bacteria detected from the upper and lower divided images as duplicately detected bacteria. The terminal 2 identifies bacteria that have been detected redundantly for each of the boundary parts (four boundaries in FIG. 22) between the divided images.

Finally, the terminal 2 totalizes the number of bacteria in the original sample image by summing the number of bacteria detected from each divided image. In this case, the terminal 2 prevents double counting by subtracting the number of bacteria identified as being duplicated above. Further, as shown in the lower part of FIG. 22, the terminal 2 displays the original specimen image with a bounding box attached thereto based on the bacterial detection results in each divided image. Note that instead of adding a bounding box to the original specimen image, a bounding box may be added to a divided image (or an enlarged image before division) and displayed.

FIG. 23 is a flowchart showing the procedure of bacteria detection processing according to the seventh embodiment. After acquiring the specimen image (step S32), the terminal 2 executes the following process.
The terminal 2 generates an enlarged image by enlarging the specimen image to a predetermined magnification (step S701). The terminal 2 divides the enlarged image into a plurality of divided images according to the magnification (step S702). The terminal 2 inputs each divided image into the detection model 50 and detects bacteria from each divided image (step S703).

The terminal 2 specifies, among the bacteria detected from each divided image, bacteria that have been detected overlappingly in two adjacent divided images (step S704). Specifically, as described above, the terminal 2 identifies bacteria detected from the boundary between two adjacent divided images and from both of the two divided images.

The terminal 2 totals the number of bacteria detected from the sample image (step S705). Specifically, the terminal 2 totals the number of bacteria detected from each divided image in step S703, and subtracts the number of bacteria identified as duplicate in step S704 from the total number. The terminal 2 moves the process to step S35.

As described above, according to the seventh embodiment, by enlarging the specimen image, the accuracy of bacterial detection can be improved.

The embodiments disclosed herein are illustrative in all respects and should be considered not to be restrictive. The scope of the present invention is indicated by the scope of the claims rather than the above-mentioned meaning, and it is intended that all changes within the scope and meanings equivalent to the scope of the claims are included.

1 Server (information processing device)
11 Control unit 12 Main storage unit 13 Communication unit 14 Auxiliary storage unit P1 Program 2 Terminal 21 Control unit 22 Main storage unit 23 Communication unit 24 Display unit 25 Input unit 26 Auxiliary storage unit P2 Program 50 Detection model 3 Detection device 31 Control unit 32 Imaging unit 33 XY-axis motor driver 34 XY-axis motor 35 LD driver 36 LD light source

Claims (13)

Obtain a sample image of a sample that can detect fluorescence,
The acquired sample image is inputted into a model that has already learned training data in which coordinate data of each of microorganisms or particulates and contaminants contained in the sample image is added, and at least microorganisms are detected. or detect particulates,
Detecting the microorganism or particulate from the specimen image by rule-based image processing or another model different from the model,
a second sample image showing the microorganisms or microparticles commonly detected in the model, the rule base or another model, and the microorganisms or microparticles detected in either one as objects in different display modes; and a list showing the number of microorganisms or particulates detected by the model, the number of detections by the rule base or other models, and the number of commonly detected microorganisms or particulates. A program to run. The program according to claim 1, wherein the specimen image is an image of the specimen stained with a fluorescent stain. Generating a document file describing the detection results of the microorganisms or particulates including the number of detections,
The program according to claim 1 or 2, which outputs a saved file including the document file and an image file of the second specimen image. the specimen is a food;
The document file that describes the detection results of the microorganisms or particles in the plurality of steps is generated by referring to the saved file when the microorganisms or particles are detected in each of the plurality of steps of manufacturing the food. The program described in 3 . After outputting the detection results of the microorganisms or microparticles, receiving an operation input for adding correct coordinate data of the microorganisms or microparticles to the second sample image;
The program according to any one of claims 1 to 4 , wherein the model is updated based on the sample image and the correct coordinate data. The program according to claim 5 , wherein input of the correct coordinate data is accepted by accepting an operation input for adding or deleting the object to the second sample image. 7. A plurality of types of microorganisms or particles are detected by inputting the acquired specimen image to each of the plurality of models that have already learned the training data according to the types of the microorganisms or particles. or the program described in paragraph 1. generating an enlarged image by enlarging the specimen image to a predetermined magnification;
The program according to any one of claims 1 to 7 , wherein the enlarged image is input to the model to detect the microorganism or particulate. dividing the enlarged image into a plurality of divided images;
inputting each segmented image into the model to detect the microorganism or particulate from each segmented image;
9. The program according to claim 8 , wherein the microorganism or particulate is detected from a boundary between two adjacent divided images and is detected in both of the two divided images. an acquisition unit that acquires a specimen image of a specimen capable of detecting fluorescence;
The acquired sample image is inputted into a model that has already learned training data in which coordinate data of each of microorganisms or particulates and contaminants contained in the sample image is added, and at least microorganisms are detected. or a detection unit that detects fine particles;
a second detection unit that detects the microorganism or particulate from the specimen image by rule-based image processing or another model different from the model;
a second sample image showing the microorganisms or microparticles commonly detected in the model, the rule base or another model, and the microorganisms or microparticles detected in either one as objects in different display modes; and an output unit that outputs a list showing the number of microorganisms or particulates detected by the model, the number of detections by the rule base or other model, and the number of commonly detected microorganisms or particulates. An information processing device comprising: Obtain a sample image of a sample that can detect fluorescence,
The acquired sample image is inputted into a model that has already learned training data in which coordinate data of each of microorganisms or particulates and contaminants contained in the sample image is added, and at least microorganisms are detected. or detect particulates,
detecting the microorganism or particulate from the specimen image by rule-based image processing or another model different from the model;
a second sample image showing the microorganisms or microparticles commonly detected in the model, the rule base or another model, and the microorganisms or microparticles detected in either one as objects in different display modes; and a list showing the number of microorganisms or particulates detected by the model, the number of detections by the rule base or other model, and the number of commonly detected microorganisms or particulates. Information processing method performed by. An information processing system comprising a detection device that images a specimen capable of detecting fluorescence, and an information processing device connected to the detection device,
The information processing device includes:
an acquisition unit that acquires a sample image obtained by capturing the sample from the detection device;
The acquired sample image is inputted into a model that has already learned training data in which coordinate data of each of microorganisms or particulates and contaminants contained in the sample image is added, and at least microorganisms are detected. or a detection unit that detects fine particles;
a second detection unit that detects the microorganism or particulate from the specimen image by rule-based image processing or another model different from the model;
a second sample image showing the microorganisms or microparticles commonly detected in the model, the rule base or another model, and the microorganisms or microparticles detected in either one as objects in different display modes; and an output unit that outputs a list showing the number of microorganisms or particulates detected by the model, the number of detections by the rule base or other model, and the number of commonly detected microorganisms or particulates.
An information processing system equipped with. Obtain a sample image of a sample that can detect fluorescence,
generating an enlarged image by enlarging the specimen image to a predetermined magnification;
dividing the enlarged image into a plurality of divided images;
For a sample image for training, each divided image is inputted into a model that has been trained with training data in which the coordinate data of each of microorganisms or particulates and contaminants contained in the sample image is attached, and at least microorganisms or particulates are added. is detected from each divided image,
identifying the microorganism or particulate that is detected from the boundary between the two adjacent divided images and detected in both of the two divided images;
Output the detection results after subtracting the identified microorganisms or particulates.
A program that causes a computer to perform a process.

Images