JP2012080802A - Method and device for fishing bacterial colony - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide, in a device which fishes many bacterial colonies cultivated by a specimen, a method for automatically or semi-automatically determining a bacterial colony fished by a cultivated petri dish, and to provide a method for determining accurate drug resistance by mixing and fishing different kinds of bacterial colonies when plural kinds of bacterial colonies existing on the petri dish, for example, when measuring the drug resistance.SOLUTION: The device for fishing bacterial colony includes: automatically extracting the isolated colony from an image illuminated from a plurality of directions (step 402); calculating a characteristic amount of image from a plurality of images illuminated from the plurality of directions (step 404); and grouping based on the characteristic amount and determining a colony to be fished based on a result of grouping (step 411).

Description

  The present invention relates to a method and apparatus for catching bacterial colonies cultured in a petri dish.

  In a bacterial test for a specimen, a method is used in which the bacteria are cultured in a growth medium in a petri dish and the bacteria are caught. In this method, a bacterial suspension is generally applied to a growth medium, cultured in a thermostatic chamber, optically observed visually, and the target bacterial colony is caught. The sterilized bacteria are used in AST (Antimicrobial Susceptibility Test) for examining resistance to antibiotics. A method for testing the resistance of bacteria to antibiotics is described in detail, for example, in Patent Document 1. The MIC (Minimum Inhibitory Concentration) method is a major method in AST. By applying bacteria collected on multiple media containing different concentrations of drugs and observing the growth of the bacteria, It is a technique to measure the minimum concentration of drugs that suppress

  On the other hand, in Patent Document 2, as a colony transfer device, a bacterial colony in a petri dish is imaged with a TV camera, and an inspector visually confirms an image projected on a monitor, and selects and instructs a colony to fish. Thus, it is described that the pickup is automatically moved to the position of the colony, the bacterial colony is caught and transplanted to a test tube or the like.

JP 63-32477 A JP 59-11173 A

  In order to perform AST such as the MIC method, a certain amount of bacteria is required, and therefore it is necessary to catch a large number of bacterial colonies formed in the petri dish. This means that if a relatively small colony is not fished, it takes a long time to cultivate the fish after it is cultured in a petri dish, and it is difficult to culture in a short time.

  In addition, in the MIC method, it is impossible to test the resistance of an accurate antibiotic unless the bacterial species of all colonies caught are the same, but until now, it depends on the visual results of multiple inspectors. There was a problem that traceability could not be obtained. In addition, it is desirable to reduce the number of inspectors in order to stabilize visual judgment that may vary, but this is a trade-off with the number of samples that can be processed per hour. Furthermore, AST needs to catch a certain amount of the same type of bacteria, and if it is mixed with other bacterial species (contamination), it will not be possible to obtain accurate drug resistance. To achieve this, it is necessary to realize a highly accurate classification function.

  On the other hand, as a colony transfer device described in Patent Document 2, a bacterial colony in a petri dish is imaged with a TV camera, and an inspector visually confirms an image displayed on the monitor, and selects a colony to fish. By instructing the above method to automatically move the pickup to the location of the colony and to pick up the bacterial colony and transplant it to a test tube etc. Since it was selected one by one, the inspector's man-hours cannot be reduced so much. In order to reduce the number of inspectors' work, it is necessary to automate the visual judgment process. However, since AST must prevent contamination as much as possible, it is classified at the same level as the inspector's judgment. A system that can sort colonies at high speed or function is required. The appearance of colonies varied depending on various factors such as culture time, culture medium, specimens, and strains, so it was very difficult to achieve a highly accurate automatic classification function.

  The present invention provides a method and apparatus for fishing a bacterial colony that can be classified with high accuracy by a simple operation.

In order to solve the above-described problems, the outline of typical ones of the inventions disclosed in the present application will be briefly described as follows.
(1) Illuminating means for illuminating bacterial colonies cultured on a culture medium stored in a container, imaging means for imaging the bacterial colonies illuminated by the illuminating means to obtain a plurality of images, and the imaging means A plurality of bacterial colonies are extracted from the plurality of images acquired by the above, and based on the feature amounts respectively calculated from the extracted plurality of bacterial colony images, classification and isolation of the plurality of extracted bacterial colonies Image processing means for performing determination, the plurality of bacterial colonies classified by the image processing means, and a bacterial colony of a fishing fungus candidate selected from the plurality of bacterial colonies isolated by the image processing means, Bacterial colonies comprising display means for displaying and fishing means for fishing at least one of the bacterial colonies of the fishing fungus candidates It is picked apparatus.
(2) The bacterial colony fishing apparatus according to (1), wherein the image processing means further includes storage means for storing a plurality of parameter recipes in advance, and information including at least medium information stored in the container. Information input means for transmitting to the image processing means, the image processing means for the plurality of parameter recipes selected based on information including at least medium information stored in the container transmitted by the information input means The bacterial colony fishing apparatus is characterized in that the plurality of bacterial colonies are classified using at least one.
(3) The bacterial colony fishing device according to (2), wherein the image processing means uses each of two or more parameter recipes selected from the plurality of parameter recipes stored in advance, The bacterial colony fishing apparatus is characterized in that a plurality of bacterial colonies are classified, and the display means displays a plurality of results classified by each of the selected two or more parameter recipes.
(4) The bacterial colony fishing apparatus according to (1), wherein the image processing unit extracts a plurality of bacterial colonies from the plurality of images, and the plurality of extracted bacterial colonies An intercolony distance measuring means for measuring the distance between them, a feature quantity calculating means for calculating a feature quantity calculated from each of the extracted plurality of bacterial colony images, and a plurality of bacterial colony images, respectively. On the basis of the feature amount and the measured distance between the plurality of bacterial colonies, the isolation determination means for performing isolation determination of the plurality of bacterial colonies, and the feature amount calculated from the images of the plurality of bacterial colonies, respectively. Based on the colony classifying means for classifying the plurality of bacterial colonies, the isolated bacterial colonies and the classified results, And picked candidate selection means for-option is picked device of bacterial colonies and having a.
(5) A detection optical system that can capture a petri dish with illumination from a plurality of directions and elevation angles, a colony region is extracted from the image by a colony extraction unit, and an external appearance of each colony by a feature amount calculation unit And color feature data, the isolated colony determination unit calculates the degree of colony isolation, and the colony classification unit performs grouping from the parameter recipes already possessed as teaching information and the feature data distribution of colonies with low isolation level, An image processing mechanism with a flow that allows the selection of colonies to be selected from the degree of isolation and feature data in the fish colony candidate selection unit, and the user can check the grouping results and the colonies to be fished, It is a device configured with a GUI that can input correction information from the user and reflect the classification results in real time.
(6) Illuminating the bacterial colonies cultured on the culture medium stored in the container, capturing the illuminated bacterial colonies to obtain a plurality of images, and obtaining the plurality of images Extracting a plurality of bacterial colonies from the image, and performing classification and isolation determination of the plurality of extracted bacterial colonies based on feature amounts respectively calculated from images of the extracted plurality of bacterial colonies; and the classification Displaying the plurality of bacteria colonies that have been isolated and the bacteria colonies that are selected from the plurality of bacterial colonies that have been isolated and selected from the plurality of bacteria colonies on a display means; And a method for fishing bacterial colonies, comprising the step of fishing at least one of the colonies.
(7) The method for fishing bacterial colonies according to (6), wherein the step of classifying and isolating the plurality of extracted bacterial colonies is based on information including at least medium information stored in the container. The plurality of selected bacterial parameter recipes are used to classify the plurality of bacterial colonies, and the displaying step includes displaying a plurality of results classified by the selected plurality of parameter recipes. This is a characteristic method for fishing bacterial colonies.

  According to the present invention, it is possible to provide a bacterial colony fishing method and a fishing device that can be classified with high accuracy by a simple operation.

It is a figure which shows the example of whole structure of the automatic fungi apparatus system regarding an Example. It is a front view which shows the schematic structure of the illumination system in an Example. It is a plane which shows the structure of the outline of the illumination system in an Example. It is a front view which shows the schematic structure of the other example of the illumination system in an Example. It is a plane which shows the general | schematic structure of the other example of the illumination system in an Example. It is a figure which shows the flowchart of the image process in an Example. It is a figure which shows the relationship between the illumination conditions of the picked-up image of the bacterial colony by the detection system of an Example, and bacterial colony shape. It is the picked-up image of the bacterial colony by the detection system of an Example. It is a graph which shows the brightness profile of the captured image of the bacterial colony by the detection system of an Example. It is sectional drawing of the petri dish which shows the captured image by (a) transmission illumination in an Example, and (b) transmission illumination direction. It is sectional drawing of the petri dish which shows the captured image by (a) low angle illumination in an Example, and (b) low angle illumination direction. It is sectional drawing of the petri dish which shows the captured image by (a) high angle illumination in an Example, and (b) high angle illumination direction. In the case of the culture medium to which the component containing a colloid was added, it is a figure which shows the scattering image of the (a) picked-up image by transmission illumination, (b) transmission illumination in a culture medium, etc. in an Example. In the case of the culture medium to which the component containing a colloid was added, it is a figure which shows the internal diffusion by the (a) picked-up image by an upward illumination in the Example, and the (b) upward illumination in a culture medium. It is a figure explaining the distance between the (a) distance between isolated colonies in an Example, and the (b) colony area | region where the several colony combined, and an isolated colony. It is a flowchart which shows the procedure which extracts the image feature-value of the bacterial colony in an Example. It is explanatory drawing of the various non-isolation colonies in an Example. It is (a) database explaining the classification method of the bacterial colony in an Example, (b) The table | surface which shows an example of the index of a database, (c) The graph which shows distribution of a feature vector. It is a graph which shows an example of distribution in the bacteria colony feature-value space in an Example. It is an image of a petri dish in which bacterial colonies are distributed in the examples. An example of the clustering system in an Example is shown, (a) The graph of the determination result of only an isolated colony, (b) The graph of the determination result of an isolated colony and a quasi-isolated colony. 5A is a graph showing a feature vector distribution of colonies determined to be isolated, and FIG. 5B is a graph showing a colony feature vector distribution up to a final selection result. It is a figure which shows an example of the GUI display in an Example. An example of classification result correction in an example is shown, (a) classification result before change by feature amount weighting, (b) classification result after change by feature amount weighting. It is a figure which shows an example of the parameter recipe in an Example, and the process result by this. It is a figure which shows the other Example of an illumination system. It is a flowchart which shows the procedure which extracts the bacterial colony candidate area | region in an Example. It is a graph explaining the brightness | luminance profoul of the bacterial colony in an Example.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows the overall configuration of the automatic fishing device according to the present invention. An apparatus configuration for acquiring an image of a cultured bacterial colony in the petri dish 107 includes an illumination unit that illuminates the bacterial colony in the petri dish 107 from above (a low angle illumination unit 101, a high angle illumination unit 102), and an optically transparent unit. A transmission illumination unit 104 that illuminates the petri dish 107 through the petri dish 107 from below and an imaging unit 103 such as a camera that obtains an image of the illuminated bacterial colony are appropriately used. Here, there is a bacterial culture medium in the petri dish 107, and a bacterial colony is cultured on the specimen to be examined, which is applied to agar and placed in an incubator for about 24H to 48H.

  The petri dish 107 is placed by a pedestal 106, and a light shielding plate 105 that can be taken in and out is disposed between the surface of the pedestal 106 on which the petri dish 107 is placed and the transmitted illumination unit 104. As the base 106, a transparent acrylic plate or glass that transmits visible light may be used. Further, as the light shielding plate 105, it is desirable to use a diffuse reflection plate having a reflection coefficient as low as possible, and a movable diaphragm, a liquid crystal shutter, or the like may be used. In the case of a movable diaphragm, it is desirable to apply a black coating process with a matte surface, and in the case of liquid crystals, an antireflection process is preferably performed. The optical system control unit 108, which will be described later, is set so that the illumination light can illuminate the petri dish 107 when the transmissive illumination unit 104 is turned on, and the illumination unit (low-angle illumination unit 101, The petri dish 107 is set to be dark when the high-angle illumination unit 102) is turned on. This makes it possible to suppress the appearance of characters printed on the bottom surface of a general petri dish 107.

  The optical system control unit 108 can turn on / off / change the intensity of the illumination in any combination of the low-angle illumination unit 101, the high-angle illumination unit 102, and the transmission illumination unit 104, and the imaging / exposure in the imaging means 103 such as a camera. It is also possible to control the change of the amount and the insertion and removal of the light shielding plate 105. The optical system control unit 108 also functions as an image input unit that acquires image information captured by the imaging unit 103, but an image input unit may be provided separately from the optical system control unit 108.

  The image processing unit 109 performs grouping by extraction of bacterial colony regions from a plurality of images acquired by the optical system control unit 108, calculation of image feature amounts from each bacterial colony image, isolation determination / classification, and the like. Here, the feature amount is constituted by, for example, quantifying the area and perimeter of the colony, color information, lightness information, background lightness, luminance, saturation, and the like. The plurality of images acquired by the imaging unit 103 are received by the image acquisition unit 126 in the image processing unit 109 via the optical system control unit 108. In the image processing unit 109, in addition to the image acquisition means 126, a colony extraction means 127, an inter-colony distance measurement means 128, a feature amount calculation means 129, an isolation determination means 130, a colony classification means 131, a fishing fungus candidate selection means 132 , Having a fishing fungus target determining means 133, and by these means, the grouped colonies of the fishing fungus target are recognized in a step described later. The petri dish information input unit 110 can read a bar code printed on the petri dish and input information such as the type of specimen and culture medium to the image processing unit 109. The secondary storage device 111 can store an image captured by the device or processed image, data, colony position, area and other information, parameter recipes learned in advance and after, and the like.

  The GUI 112 displays captured images, colony extraction results, isolated colony determination results, colony classification results, feature amount histograms, and the like. When it is desired to correct the classification result, the user can input parameter changes and teaching information via the GUI. The result of newly inputting and correcting the information is displayed on the GUI in real time. In addition to the mode that is confirmed by the GUI, an automatic operation mode is also provided. In the automatic operation mode, the colony selected by the system is fished as it is without confirmation from the inspector.

  The petri dish transport means 113 stores the imaged petri dish 107 in the stacker 114, and the petri dish may be transported after grouping by the image processing unit 109 or before grouping. When it is necessary to check the grouping result by the inspector, this work time often takes a relatively long time. Therefore, in order to increase the throughput of the apparatus, the inspector's confirmation work should be performed after the image capturing is completed. It is desirable that the next petri dish image input operation is performed without waiting for the inspection, and in parallel with this, the inspector uses the acquired image to check and correct the grouping result, determine the colony to be fished, and the like. Similarly, the transport means 115 also transports the petri dish 125 stored in the stacker 114, and is used to transport the petri dish 125 from the stacker 114 to the fishing germ area.

  The configuration of the device that picks up the desired colony from the bacterial colonies in the petri dish 107 includes a fishing fungus needle 117 for picking up the colony, a Z stage 118 and an XY stage 119 for scanning the fishing fungus needle 117, and the petri dish 125 upward. The upper illumination unit 121 that illuminates from below, the transmission illumination unit 120 that illuminates through the petri dish 125 from below, and an imaging means 122 such as a camera that obtains an image of the illuminated bacterial colonies are appropriately used. The Here, the stage control unit 116 controls the Z stage 118 and the XY stage 119 that scan the fishing fungus 117 and the above-described transport means 113 and 115.

  The petri dish 125 transported from the stacker is illuminated by the transmission illumination unit 120 and the upper illumination unit 121, and the image can be taken using an imaging means such as a 122 camera. The image can be taken in via the image input means of the control unit 108. The image input to the optical system control unit 108 is compared with the image acquired by the camera 103 in the image processing unit 109, and the positional deviation when the same petri dish is conveyed through the stacker 114 is corrected. For example, the captured image of the imaging unit 103 is I0, and the captured image of the imaging unit 122 is I1. Note that the captured image I0 and the captured image I1 are photographed so that the illumination conditions are the best in the two optical systems. Since the petri dish is photographed brightly by setting the petri dish background black, it is possible to extract the petri dish region by binarization processing. This is obtained in each of the captured image I0 and the captured image I1, and the center of rotation of the petri dish is roughly aligned. In each of the captured image I0 and the captured image I1, rotation-invariant feature point detection such as SURF is performed, and a correspondence relationship between the detected feature points between the captured image I0 and the captured image I1 is obtained. From the correspondence of the obtained feature points, a rotation parameter R and a translation parameter T for correcting the deviation between these images are obtained by an ICP algorithm or the like, and coordinates at the time of fishing are obtained. Note that the coordinate alignment method is one embodiment, and other methods such as a method of calculating the amount of rotation deviation using Fourier transform of an image, a method using normalized correlation calculation, and the like may be adopted. .

  A microplate 123 having a plurality of wells is arranged in a range where the fishing fungus needle 117 after the fishing fungus is scanned or provided separately. In each well, for example, Ca ion 50 mg / l, Ca ion 25 mg / l and a drug for evaluating bacterial resistance are added to Muller Hinton culture solution so as to have different concentrations in each well. Other liquid media may be used, or a solid medium may be used. By controlling the Z stage 118 and the XY stage 119, the fish needle 117 is moved to the coordinates of the bacteria to be fished in the petri dish 125, and the colony is picked up. To pick up a colony, a force sensor should be incorporated at the location where the fishhook is attached so that the point where the fishhook contacts the solid medium can be detected. After this, the bacterial colonies that have been caught are placed in a container containing a liquid such as physiological saline or liquid medium. This operation is performed on several bacterial colonies, and after the bacterial colony concentration is sufficient, for example, to exceed the set value, any of the microplates 123 is suspended using a suspended liquid 124 such as physiological saline. Put a colony in that well. The fishing fungus 117 can be changed to a different one each time a different colony type is picked up. If the amount of colonies formed in the petri dish is insufficient, and one petri dish cannot cultivate bacteria in all the number of wells set in the microplate 123, the specimen bacteria can be cultured in multiple petri dishes, A specific bacterial species is determined from the petri dish, and the bacteria may be transferred from the plurality of petri dishes to one microplate. This makes it possible to execute the MIC method even with a relatively short culture time. In this way, when selecting colonies to be picked from captured images of a plurality of petri dishes, a plurality of petri images of the same specimen can be easily displayed on the GUI 112.

Next, details of each component will be described.
FIGS. 2A and 2B show configuration examples of the illumination means of the low-angle illumination unit 101 and the high-angle illumination unit 102 as seen from the side and top. The low-angle illumination unit 101 is composed of four divided light sources 2011, 2012, 2013, and 2014, and lighting control can be performed individually using the optical system control unit 108. Similarly, the high-angle illumination unit 102 includes four divided light sources 2021, 2022, 2023, and 2024, which can also be individually controlled by the optical system control unit 108. Since the surface of the bacterial colony is relatively smooth, the area that directly reflects is brightly photographed by lighting.

  As shown in FIG. 2, the brightness in the plane including the illumination, the center of the colony, and the camera optical axis can be obtained by performing illumination for each direction. When obtaining the detailed shape of the bacterial colony, the optical system control unit 108 is controlled to illuminate the divided light sources 2011, 2012, 2013, and 2014 one by four, and further, the divided light sources 2021, 2022, It is desirable to take four images of 2023 and 2024 one by one, for a total of eight times. However, in order to shorten the shooting time, it is desirable to shoot several of them simultaneously. On the other hand, in order to shorten the photographing time, for example, one image that simultaneously illuminates the divided light sources 2011, 2012, 2013, and 2014, and one image that simultaneously illuminates the divided light sources 2021, 2022, 2023, and 2024 It may be a single image.

  When direct reflected light is detected, there is a problem that the color information of bacterial colonies in the vicinity is lost. Color information is important to classify bacterial colonies by type. Therefore, we decided to select an angle where the reflected light is not detected as much as possible in low-angle illumination. The illumination of the low-angle lighting unit 101 is the highest angle θ at the edge of the petri dish. However, as a result of evaluation, most colonies can obtain color information without being directly influenced by reflected light. An angle is desirable.

  In addition, even in high-angle illumination, the edge of the petri dish is illuminated at the highest angle, θ2, but if this angle exceeds 80 °, the medium tilts at the end of the petri dish due to surface tension, It was found that the medium itself may be detected brightly by direct reflection. Therefore, it is desirable to set the high-angle illumination so that the illumination angle does not exceed 80 ° even if it is the highest-angle illumination.

  By setting the illumination angle in this way, the low-angle illumination consisting of the split light sources 2011, 2012, 2013, 2014 does not detect direct reflected light in most cases. There is not much information to be obtained. For this reason, it is desirable from the viewpoint of shortening the photographing time that the low-angle illumination is taken with simultaneous illumination from all directions. In addition, you may make it image by carrying out simultaneous illumination from the arbitrary some directions determined by the user's selection etc. instead of omnidirectional simultaneous illumination. FIG. 3 shows a configuration similar to that shown in FIG. The illumination of the low-angle illumination unit 101 is configured using the divided light sources 3011, 3012, 3013, and 3014, and the illumination of the high-angle illumination unit 102 is configured using the divided light sources 3021, 3022, 3023, and 3024. Each of these can be controlled individually using the optical system control unit 108. Here, in FIG.2 and FIG.3, although the example provided four each as a division | segmentation light source of a low angle illumination unit and a high angle illumination unit was shown, it is not restricted to this, Each using arbitrary some division | segmentation light sources You may comprise a lighting unit. Furthermore, the present invention is not limited to the combination of the low-angle illumination unit and the high-angle illumination unit, and a medium-angle illumination unit may be provided at an angle between these to improve the angle detection resolution.

  Considering the classification based on the direct reflected light of the colony, it is necessary to detect the direct reflected light from almost the same normal direction of the colony at any position of the petri dish 107. However, the petri dish is usually about φ90 mm, and in order to realize this, the high-angle illumination unit 102 and the imaging means 103 such as a camera need to be separated greatly from the petri dish. In order to make the necessary distance as small as possible, the camera may be a telecentric optical system. By adopting this method, the distance between the lens of the imaging system and the petri dish can be reduced.

  As for the illumination system, the position of the illumination and the petri dish can be shortened by adopting a method in which the high-angle illumination unit 102 is made up of a large number of LEDs and each LED is provided with a microlens to make it relatively parallel light. Can do. As another configuration, there is a method shown in FIG. 2001 and 2003 are light sources, 2002 and 2004 are parabolic mirrors, respectively, and the light source 2001 and the light source 2003 are arranged at the focal positions of the parabolic mirror 2002 and the parabolic mirror 2004, respectively. As a result, light emitted from the light source 2001 and the light source 2003 toward the parabolic mirror 2002 and the parabolic mirror 2004, respectively, is reflected by these parabolic mirrors and becomes parallel light and is irradiated on the petri dish 107. . As a result, illumination can be performed from almost the same direction at an arbitrary position of the petri dish 107. The mirror is not limited to the parabolic mirrors 2002 and 2004, and a spherical mirror or the like may be used as appropriate.

  Here, the internal flow of the image processing unit 109 shown in FIG. 4 will be described using each functional means provided in the image processing unit 109. A plurality of images sent from the optical system control unit 108 are received by the image acquisition means 126 (step 401). The acquired plurality of images are sent to the colony extracting means 127, and the colony region of the entire petri dish is extracted by combining the plurality of images (step 402). The method of combining images is determined by the type of medium obtained from the petri dish information input unit 110. For example, in a tripty case soy agar medium that transmits visible light, the brightness of the colony region present in the petri dish illuminated by the transmitted illumination unit 104 is significantly lower than the surrounding area. Therefore, at the time of extraction, the area is obtained by increasing the weight of the transmitted illumination image as compared with other illumination images. In contrast, in a blood agar medium in which visible light is scattered internally, the colony area is not revealed because visible light from transmitted illumination does not pass through the medium. However, a petri dish illuminated by low-angle / high-angle illumination is not visible in the medium area. Since the brightness of the colony area is often revealed, the area is obtained by increasing the weight of these upward illumination images when performing colony extraction.

  Next, the distance between the colony distance measuring means 128 obtains the distance between the individual colony regions extracted by the colony extracting means 127 and the colonies adjacent to them (step 403). On the other hand, the feature quantity calculation means 129 calculates feature quantities such as the area, perimeter, brightness, and saturation of each colony determined by the colony extraction means 127 (step 404). In the isolation determination unit 130, whether the colony is formed in an isolated manner from the inter-colony distance obtained by the inter-colony distance measurement unit 128 and the appearance feature amount obtained by the feature amount calculation unit 129 with respect to the extracted colony Is determined (step 405). The isolation determination by the isolation determination means 130 is made on the basis that the distance between colonies is equal to or greater than a certain threshold and that two or more colonies are not combined. The former condition only needs to determine whether the value obtained by the inter-colony distance measuring means 128 is equal to or greater than a threshold value. The latter condition needs to be determined by processing the feature amounts calculated by the feature amount calculation means 129 in an integrated manner. For example, when two colonies with the same diameter are tightly coupled, they often have an elliptical shape with concatenated portions. Therefore, the moment feature is obtained, the major axis and minor axis of the colony shape are calculated, and the aspect ratio is calculated. It can be determined by. However, when 3 or more colonies of the same diameter or 2 large and small colonies are combined, it cannot be judged under this condition. Therefore, an integrated determination is made from the obtained feature quantity using teaching learning.

  Further, the isolation determination means 130 has two parameter sets. One parameter set isolates only colonies that are unlikely to be contaminated even if they are fished. The colony information determined by this parameter is transmitted to the fishing fungus candidate selection means 132. In order to obtain a large amount of sample data used at the time of classification, the remaining parameter sets are set to judgment conditions looser than the previous parameter set, and an isolated colony and a quasi-isolated colony that can be used at the time of classification are determined. The result based on the parameters for the fishing fungus and the result based on the parameter for classification are displayed on the GUI 112 at the same time, and the user can correct both parameters while viewing this. The colony information determined by this parameter set is transmitted to the colony classification means 131.

  In the colony classification unit 131, teaching classification or clustering is appropriately performed using the distribution of the isolated colony determined by the isolation determination unit 130 and the distribution of the feature amount vector calculated by the feature amount calculation unit 129 and the information obtained from the petri dish information input unit 110 as appropriate. Colony grouping is performed by the method (step 406). Even if a large number of isolated colonies are formed on the petri dish, it is not necessary to fish all the colonies because only the amount of colonies necessary for the inspection needs to be collected. Therefore, the fishing fungus candidate selection means 132 selects the fishing fungus candidates by narrowing down to colonies that are less likely to be contaminated by the isolation determination by the isolation determination means 130 and colonies near the center of the group in the grouping result by the colony classification means 131 ( Step 407) thereby improving the final fishing fungus accuracy. The colony classification result and the fish selection candidate result are displayed on the GUI 112 as a petri dish with marking (step 408), and the user can check the result. Here, if a colony that is not subject to fishing fungus is selected as a candidate for fishing fungus, or if many colonies that are not subject to fishing fungi are determined to be fishing fungi, The user can modify the result. If there is a slight modification, the parameters of the fish selection candidate 132 may be modified. If the modification is desired, the parameters of the isolation determination means 130 may be modified. When the modification input from the user is received via the GUI 112, In this case, the calculation is again performed, and the fishing fungus candidate selection means 132 selects the fishing fungus candidate (step 409). In addition, if there is a colony that is incorrectly grouped with respect to the classification result, the user selects the colony with the GUI 112 as necessary, receives the input of classification correction from the user, and again with the colony classification unit 406. A classification calculation is made (step 410). In addition, if an inappropriate colony that greatly affects the classification result is included in the colonies to be classified, the parameters of the isolation determination means 130 are received as necessary according to the input of correction from the user. The classification result is corrected (step 410). If there is no correction input in the GUI 112, the final fishing target is determined by the fishing target determination unit 133, and the determination information is transmitted from the fishing target determination unit 133 to the secondary storage device 111 and stored. (Step 411).

  FIG. 5 shows the manifestation of bacterial colony features by each illumination. In high-angle illumination, the angle of the bacterial colony at the position of the reflected light can be obtained from the position of the reflected light. The normal direction of the bacterial colony at the position of the bacterial colony from which the direct reflected light was obtained is obtained by using the unit vector Vc from the reflected light position to the camera lens direction and the unit vector Vi from the reflected light position to the illumination (Vc + Vi). Expressed with / 2. If the colony is flat, it can be estimated that the position where the reflected light is detected is near the center of the colony, and if the reflected light position is near the colony, the colony has a height from the medium. Further, when the colony has a shape broken from the dome shape, the reflected light is detected at a plurality of positions. On the other hand, in low-angle illumination, even if the surface shape of the colony changes slightly, the brightness does not change, and it is easy to acquire the color and brightness information of the surface of the bacterial colony.

FIG. 6 shows the distribution of bacterial colonies in the petri dish. On the petri dish 601, there is an area 602 that appears slightly brighter due to the effect of surface reflection on the petri dish and surface tension at the edge of the petri dish near the outside of the petri dish. There is also a region 603 where the lightness or color of the medium changes due to the secretion of bacterial colonies. 604 is a small-sized bacterial colony having a relatively low contrast from the medium, and 605 is a high-contrast bacterial colony. Bacterial colonies with high contrast are easy to detect, but 603 colonies have low contrast, and the areas such as 602 and 603 are farther away from the average brightness or color of the medium. If a colony is detected by the difference in color or brightness from the average of the medium, a region that is not a colony such as 602 or 603 is also extracted as a colony under the condition that 604 can be extracted. Therefore, in this embodiment, an isolated colony can be extracted using the following three steps. That is,
(I) A spatial bandpass filter is applied to remove low frequency components that are characteristic of 602 and 603.
(II) Based on the local brightness around the area of interest, the brightness or color that appears to be a medium is calculated, and based on the comparison with the local brightness or color, it is extracted as a colony candidate if different To do.
(III) The brightness or color distribution of the region extracted as a colony candidate is evaluated, and a case where the brightness distribution is similar to a dome shape is determined as a colony.

  FIG. 7 shows the concept of the algorithm of step (II). FIG. 7 shows a one-dimensional brightness distribution of a photographed image. Although 701 and 702 are colonies to be extracted, if a simple binarization method is employed, the region of 703 is extracted as a colony. Therefore, when determining the presence / absence of a colony in the region 701, the presence / absence of a colony is determined based on the region 704 in the vicinity when determining the presence / absence of the region 703 and the region 702 in the vicinity. Is good.

As the method (III), for example, quadratic function fitting can be mentioned. In the process of (II), even if a bright region is extracted based on the vicinity of the colony to be extracted, there is a possibility that a region where a plurality of bacterial colonies overlap is extracted, for example, as indicated by 606 in FIG. is there. Multiple colony assemblies may not consist of a single bacterium and must be removed. When the transmission image brightness of the candidate area of the colony obtained in (II) is I (x, y), for example,
(Equation 1) Error = Σ (Ax ² + Bx + C + Dy ² + Ey + F + Gxy + H−I (x, y)) ²
Is calculated from the colony candidate area, and the coefficients, A, B, C, D, E, F, G, and H are calculated by the least square method so that the error is minimized. In this case, it is possible to adopt a method such as making an isolated colony only when Error / S is equal to or less than a threshold value. Further, this quadratic function fitting may be similarly calculated by Gaussian function approximation.

  The step described in (II) presupposes that the image of the medium does not change sharply, and that the sharp change in brightness in the image is due only to bacterial colonies. However, there are actually other brightness changes. The biggest problem is the characters written in the petri dish. FIG. 8A (a) is an image 801 of the petri dish 107 taken by illuminating the petri dish 107 from below as shown in FIG. 8A (b) using only the transmission illumination unit 104, and FIG. 8B is an image of the petri dish 107 photographed by illuminating from a low angle as shown in FIG. 8B (b), and FIG. 8C (a) is a high angle as shown in FIG. 8C (b) in the high angle illumination unit 102. It is the image obtained by illuminating from. In general, the bottom of a petri dish used for culturing bacteria is printed with a barcode or the like for checking its traceability. This print can be seen most clearly from an image 801 in FIG. 8A (a), for example. This bar code is automatically read by the petri dish information input unit 110 when the petri dish 107 is sent to the base of the light shielding plate 105, and sample information, medium information, and the like can be obtained. Also, in hospitals and testing institutions, in order to distinguish a large number of similar bacterial culture dishes, specimen IDs may be written with magic ink or the like. As described above, the letters on the petri dish bottom may be clearer than the colonies.

  In an image illuminated from a low angle such as the image 802 of the petri dish 107 in FIG. 8B (a), this print is hardly visible, but in the case of a very transparent medium, it may be visible. The image 803 of the petri dish 107 in FIG. 8C (a) is not clear compared with the image 801 in FIG. 8A (a), but can be clearly seen in comparison with the image 802 in FIG. 8B (a). 804 is a bacterial colony, 805 is a petri dish print, and 806 is a culture medium.

  When illuminated from below as in the image 801 of the petri dish 107 in FIG. 8A (a), the print 805 blocks light, so that an image is captured very well. In the case of the low-angle illumination of the image 802 in FIG. 8B (a), most of the light is reflected by the surface and does not enter the medium 806, so that the print is hardly visible. Since the image 803 in FIG. 8C (a) is illuminated from above, the light enters the medium 806 as compared with the low-angle illumination, and the print looks clear. Here, in cases other than the transmission illumination using the transmission illumination unit 104, the light shielding plate 105 is arranged to make it difficult to see the print in the image 803 in FIG. It is difficult. Therefore, in the step (II), it is first necessary to identify the mark area which becomes an obstacle when recognizing the brightness and color of the culture medium 806 by local area processing such as printing on the petri dish bottom or characters and labels. Become.

  Here, an algorithm for identifying this mark area and excluding it to extract colony area candidates will be described with reference to FIG. First, a dark region A is extracted from a transmission illumination image obtained by photographing the petri dish 107 from below using the transmission illumination unit 104 and photographed by the imaging means 103 (step 2101). This dark area A is generally a bacterial colony area or a mark area at the bottom of a petri dish. Next, in the area excluding the A area, either the low-angle illumination unit 101 or the high-angle illumination unit 102, or a combination of a plurality of them, the petri dish 107 is illuminated from above, and the upper illumination obtained by imaging with the imaging means 103 The average brightness of the medium is obtained from the image (step 2102). Here, an image by low-angle illumination using the low-angle illumination unit 101, an image by high-angle illumination using the high-angle illumination unit 102, or both may be used. This is because there are colonies in which the image from the high-angle illumination is more obvious, and in other cases, the colony is more obvious from the image in the low-angle illumination. The most robust algorithm should do both. Furthermore, when the low-angle illumination unit 101 or the high-angle illumination unit 102 is illuminated according to the illumination direction, it is more preferable to calculate the average combined image of images illuminated from a plurality of directions or all directions, respectively. It is possible to detect bacterial colonies that do not depend on.

  In general, even when the culture medium is discolored by bacterial secretions, there is no significant discoloration. Therefore, local brightness and color of the medium are obtained from the area having brightness and color close to the average brightness (step 2103). Next, an area having a large difference from the local brightness and color is extracted as a colony candidate area (step 2104). The colony region is determined by using the method described in (III) above for the colony candidate region extracted in this way.

  Next, another colony region extraction method suitable for media having different properties will be described with reference to FIGS. 9A and 9B. For example, in the case of a medium to which a component containing a colloid such as a blood agar medium is added, as shown in FIGS. 9A (b) and 9B (b), inside the medium 806, as in 903 and 904 Visible light is scattered. Therefore, an image 901 of the petri dish 107 shown in FIG. 9A obtained by imaging in a state illuminated by the transmitted illumination of the transmitted illumination unit 104 becomes an image in which the boundary of the colony is blurred, and the colony region is not easily revealed. . On the other hand, since the image 902 of the petri dish 107 shown in FIG. 9B (a) obtained by imaging in the illumination state with the upper illumination is hardly incident on the culture medium and reflected from the petri dish bottom, the light toward the camera is hardly generated. Since the print on the back of the petri dish does not appear, it is more effective for colony extraction than a medium that transmits light. In addition, the light 905 incident on the inside of the medium 806 is randomly diffused internally and the medium surface has a uniform color, so that it can be said that it works preferentially in extracting colonies over a transparent medium. In the case of such a medium that does not completely transmit light, it is better to extract colonies mainly using an image of upward illumination. Therefore, in the secondary storage device 111, for example, an extraction parameter that is set for each type of culture medium to which an image captured by which illumination is mainly used is stored as a part of the parameter recipe, and the petri dish information input unit An optimum extraction parameter is selected from the petri dish information acquired at 110 using the GUI 112 or the like.

  Next, details of the inter-colony distance measuring means 128 for obtaining the distance between the extracted individual colonies and the colonies adjacent to them will be described with reference to FIG. When the distance between colonies of the colony 1001 is measured, the colony 1002, 1004, 1005 existing around the colony 1001 is determined to have the shortest distance between the edge of the colony 1001 and the edge of another colony. In the example of FIG. 10A, the distance 1003 to the colony 1002 among the colonies 1002, 1004, and 1005 is the shortest. Since the colony is almost circular, the radius and center are obtained from the area of the colony region. The distance between these colonies can be calculated by subtracting the radii r1 and r2 of each colony from the distance between the center of the colony 1001 and the center of the colony 1002 when the distance is 1003. When obtaining the inter-colony distance in the colony 1006 shown in FIG. 10B, the most adjacent colony may not be circular like 1008 in which a plurality of colonies are combined. In such a case, a point P at which the distance between an arbitrary point on the contour of the colony region 1008 and the center of the colony 1006 is the shortest is obtained, and the radius is calculated by subtracting the radius of the colony 1006 from that distance.

  Next, the step of extracting the image feature amount of the colony from the extracted colony region candidates will be described with reference to FIG. First, a colony 1005 is extracted from the extracted colony region candidates by the method (III) (step 1111), and an image feature amount of the colony for grouping the colonies is calculated from the extracted region for each colony (step 1112). ). As image features, high-angle illumination image 1101 (illuminated by high-angle illumination unit 102), low-angle illumination image 1102 (illuminated by low-angle illumination unit 101), and transmitted illumination image 1103 (illuminated by transmission illumination unit 104) are processed. At least the following feature amount is calculated. Note that the high-angle illumination image 1101 can be obtained by illuminating individually from a plurality of directions, sequentially obtaining a plurality of high-angle illumination images, and individually processing them, thereby obtaining a feature with excellent discrimination.

(a) Surface irregularities: 1107
As shown in FIG. 5, the surface irregularities can be obtained by calculating the normal direction of the colony at the observed position based on the position of the directly reflected light.
(b) Texture: 1107
The texture is obtained on the basis of the direct reflected light, as with the surface irregularities. If the surface is rough, a small area of direct reflected light is observed in the astigmatism with respect to the colony center for each orientation, and this is used.

(c) Color: 1108
The color feature is calculated using an image illuminated from a low angle or a transmitted illumination image. In addition, an image illuminated from a high angle may be used as an auxiliary. As the color space, a Lu * v * space, an HSV space, or the like may be used in addition to normal RGB. In particular, when using a transillumination image, the brightness varies depending on the height of the colony from the culture medium when the RGB image is used. There is a high possibility that they will not be independent. For this reason, the Lu * v * color space and HSV color space, which have independent brightness and color, have significant advantages.

(d) Shape (size, circularity, long axis / short axis): 1109
For the colony 1106 selected from the detected colonies 1105, the area size, the circularity, and the long axis / short axis when the shape is approximated to an ellipse are calculated from the image. Circularity is determined by dividing the perimeter of the region by the square root of the area. This circularity makes it possible to reveal what the bacterial edge is like an amoeba.

(e) Lightness, color, transmittance, 3D shape: 1110
The brightness is obtained using the transmitted illumination image 1103. Based on the direct reflection of the brightness and high-angle illumination image, the transmittance and height from the medium can be obtained. FIG. 22 shows the relationship between the high-angle illumination image and the transmission illumination image. 2201 indicates the brightness level at each position of the high-angle illumination image illuminated from one direction, and 2202 indicates the brightness level at each position of the transmitted illumination image.

Transmitted illumination can be used only when the medium has a high light transmittance, but it becomes darker at the colony location and becomes darker as the colony thickness increases. If the lightness inside the colony is IC (x, y) and the average value of the lightness of the medium is IM, the thickness D (x, y) of the colony at a certain (x, y) position is Desired.
(Equation 2) D (x, y) = − G (log IC (x, y) −log IM)
Here, G is a gain determined for each colony type.

It is assumed that reflected light by high-angle illumination can be detected at a certain position (X, Y). It is assumed that the normal vector at this time is inclined by θ from the vertical direction. At this time, the change in thickness when the position is shifted by Δx is −Δx tan θ. Here, when the difference of D (x, y) is calculated,
(Equation 3) D (X + Δx, Y) −D (X, Y) = − G (log IC (X + Δx, Y) −log IC (X, Y)) = − Δx tan θ
Is established. That is,
(Equation 4) G = Δx tan θ / (log IC (X + Δx, Y) −log IC (X, Y))
Thus, the thickness of the colony is obtained at an arbitrary position. Therefore, in the case of a medium with relatively high permeability, if a spot where direct reflected light is generated is found, the three-dimensional shape and volume of the colony can be obtained. Become. For example, from this information, the maximum height, volume, average height, and outer height of the colony are calculated. A vector having these features as elements is calculated, and this is used as the image feature amount of each colony.

  Next, an isolated colony determination process that is performed by appropriately using the distance between colonies extracted as described above and the calculated feature amount of the colony will be described with reference to FIG. An isolated colony here means that when a bacterium grows from a colony on a grain to form a round colony, it grows without contact with other colonies and is at a certain distance from other colonies. Defined as separated. What is necessary is just to determine about the distance between colonies using the result in the distance measuring means 128 between colonies.

  Here, a method for determining what a colony originally isolated in contact with during the growth process is handled. 1201 and 1202 are examples in which two colonies are in contact with each other or one part is joined. These combined colonies are determined to be isolated if the aspect ratio of the major axis and the minor axis obtained by the feature quantity calculation means 129 (major axis 1208 and minor axis 1209 in the example of 1201) is larger than a threshold value. be able to. When three colonies of approximately the same diameter as shown by 1203 are combined, or when two colonies with extremely different sizes are combined as shown by 1204, the determination using the above aspect ratio cannot be made. Therefore, in such a case, a feature value for isolation determination is performed so as to determine these colonies.

First, for 1203 and 1204, the statistic of k-curvature C (i) is obtained from the boundary point sequence (x _i , y _i ) (i = 1, 2,...) Of the colony region as a histogram. If multiple colonies do not join and the shape is close to a perfect circle, the curvature of any boundary point becomes constant, so C (i) of the boundary point sequence is voted for only one class. On the other hand, in the case of a colony such as 1203, the curvature in the point sequence of the circular portion is constant, but the curvature greatly changes at the boundary portion of the joining portion because of the constricted shape. When the histogram of C (i) is calculated, it has a large peak for one class and a small amount of frequency for the other class. In the 1204 colony, there is an arc portion with a different diameter from the constricted portion at the boundary, so the histogram of C (i) has two peaks and other small number of frequencies. In order to obtain the isolated determination from these histograms, the conditions may be classified by the normalized entropy of the histogram. Since the curvature histogram of an isolated perfect colony is voted only in one class, the entropy is close to zero. On the other hand, the histogram of 1203 and 1204 colonies has a high degree of randomness, such as having a number of peaks and classes other than peaks. For this reason, the entropy is close to 1, so this property can be used for determination.

  A colony of 1205 shows a close association of two bacterial colonies of different brightness. Such a colony is almost a perfect circle and does not have a constriction at the boundary of the colony. Therefore, the centroid of the colony area and the centroid of the lightness of the colony area are obtained and compared. If there is a bias in the lightness distribution, the positions of the two centroids are greatly shifted. When a small colony is formed inside a large colony of 1206, or when two colonies with almost the same brightness as in 1207 are closely connected and are almost circular, only one feature is used. It is difficult to make an isolated determination. Therefore, these colonies can be isolated by checking on the GUI 112 and comprehensively judging clues such as slight changes in brightness, contour distortion, and direct reflected light shape. is there. Specifically, for the 1206 and 1207 colonies, the feature vector of known colonies corresponding to 1206 and 1207 is collected in advance with respect to the feature vector calculated by the feature calculator 129. Judgment is performed using teaching classification such as K-NN method and SVM.

  Next, two determination methods for isolation determination will be described with reference to FIG. The isolation determination means 130 has two parameters and outputs two isolation determination results. One of the parameters is set to a severe judgment that many colonies are repelled in order to narrow down the colonies used for the fishing fungus, and the result is sent to the isolation judgment means as an isolated colony. It is done. In FIG. 15A, 1501 indicates a bacterial colony on the petri dish determined by this parameter, and 1502 indicates the feature vector. These bacterial colonies are very effective in performing a bacterial identification test because there is little risk of contamination, but if there are few colonies that satisfy this condition, the number of feature vector samples in the feature space is small. Therefore, there is a high possibility that clusters will not be formed well when classifying bacterial colonies using clustering techniques. Therefore, in order to stabilize the classification, it is desirable to increase the feature vector samples using colonies that are not subject to fishing fungi. For example, the feature vector of a colony that has been repelled in a determination based on a parameter that makes a strict determination due to a short distance between colonies such as 1503 in FIG. 15B is one of samples that form a group such as 1504 It can be one. In addition, bacterial colonies such as 1204 and 1206 shown in FIG. 12 cause contamination, but the feature vectors are close to colonies with the same diameter, and these can be used effectively at the time of classification. Therefore, the other parameter for isolation determination also determines quasi-isolated colonies that have problems with isolated colonies and fungi but can be used for classification, and these colonies are transmitted to the colony classification means 131. This makes it possible to obtain a sufficient number of feature vector samples to stabilize the result of the clustering technique.

  Next, a method for classifying colonies by the colony classification unit 131 using the determined isolated colonies and the calculated distribution of feature vectors as appropriate will be described. There are basically two methods for classifying colonies by bacterial species. One is a method for dividing each type of bacteria already registered, and the other is a method for dividing each type of colony only by a given petri dish colony image.

  The first method will be described with reference to FIG. FIG. 13A shows a feature quantity database (DB) 1301, and the database can search for data using an index 1302 shown in FIG. 13B. The bacterial species 13021 is not necessarily a single bacterial species, and may be a group of several bacterial species, but even in that case, the bacterial species 13021 is a group having a similar appearance. Next is medium 13022. Bacterial colonies may have limited growth depending on the medium, and depending on the medium, the color of the bacterial colony may vary greatly depending on the bacterial species. For this reason, information about the culture medium is extremely important in classifying bacteria. In the sample 13023, bacteria that are likely to be generated vary depending on whether the sample from which the bacteria have been acquired is, for example, the blood of the patient, feces, or urine. This information is also important in order to avoid overlooking important bacteria in classifying bacteria. Further, the culture condition 13024 of the bacterial species 13021 is also important information for classifying the bacterial species. Finally, feature amount data 13025 is described. The feature amount data 13025 is vector data, and is composed of a set of vector data of a large number of bacterial colonies of the same type. For example, reference numerals 1303, 1304, and 1305 shown in FIG. 13 (c) denote vector groups of colonies of different bacterial species photographed under the same medium, specimen, and culture conditions, respectively. If the feature vector obtained from a bacterial colony of a new unknown bacterial species is 1306, since the most similar feature vector 1305 is adjacent in the present invention, 1306 is also the same bacterial colony as 1305. Judge.

  A typical bacterial colony is registered in the database in this way, but there are cases where bacteria cultured from the specimen are not necessarily registered in the database. In order to cope with this case, there is also a method in which feature vectors obtained from the petri dish by image processing are grouped interactively without using a database.

  This algorithm will be described with reference to FIGS. 14A and 14B. In the feature amount space 1401 in FIG. 14A, reference numerals 1403, 1404, and 1405 denote colony feature amount vectors corresponding to 1406, 1407, and 1408, respectively, in the petri dish image 1402. When the inspector teaches the system that typical examples of bacterial colonies to be grouped are 1403, 1404, and 1405 when the feature space diagram shown in FIG. 14A is displayed on the GUI 112, the colony The classification unit 131 groups the feature amount vectors using a generally known clustering method or the like. As a clustering method, for example, a method such as an EM algorithm, k-means, or fuzzy k-means can be used, and the instructor's teaching result may be used as the initial grouping state before executing this algorithm. . Instead of a clustering algorithm, a general classification algorithm, for example, a classification algorithm such as Nearest Neighbor method or Naive Bayes method may be used. When a classification algorithm is used, it is executed assuming that there is a class in which samples are taught one by one.

  Further, the colony classification means 131 excludes the classification result from the grouping as an unknown colony when the feature amount is greatly different from the typical bacterial colony feature taught by the inspector. Furthermore, those that are approximately equidistant from the feature quantities of a plurality of colonies taught as typical bacterial colonies are excluded from the grouping as colonies whose grouping destinations are unknown. This is indispensable, for example, to prevent the evaluation of drug resistance using bacteria that were not originally envisaged when carrying out the MIC method. Further, the system selects a sample to be fished based on the grouping result. When a large number of colonies are cultured, it is desirable to select the grouped results so that the one close to the characteristic amount to the one selected by the inspector as a typical colony is caught.

  The result processed by the colony classification unit 131 is displayed on the GUI 112. The inspector confirms the grouping result shown in the GUI 112, and if it is a desirable classification result or a fishing fungus result, instructs the system to fish according to this result. If the grouping result is insufficient, a new bacterial colony of a specific bacterial colony is taught to the system via the GUI 112. The colony classification means 131 adds this as new teaching data of the Naive Bayes method, for example, and executes classification, and shows the grouping result to the inspector in almost real time. In addition to this, it may be handled as new teaching data of K-NN classification, or a group of teaching data added by an inspector may be reflected in the initial state of K-means clustering. Other known classification algorithms or clustering algorithms may also be used. The instructor can indicate the classification result after adding the teaching data, and the inspector can immediately determine whether the grouping result should be satisfied. To.

  There are also colonies for which inspectors cannot identify bacterial species. In such a case, it is registered in the system via the input / output terminal as a colony whose bacterial species cannot be specified. The system identifies, for example, the two closest groups in the feature space of the colony for which the bacterial species cannot be identified, and moves the boundary between the two groups to the center of the group, respectively. So that colonies are not grouped.

  This grouping result is not always performed with only one petri dish. This is because a large number of wells are arranged on the microplate, and there is a case where all wells cannot be filled with bacteria with only one petri dish, and there are cases where bacteria of one specimen may be cultured in a plurality of petri dishes. . In this case, it is preferable to display a plurality of petri dish images at the same time so that the inspector can check the plurality of petri dish images and select a teaching sample. Alternatively, one of the plurality of petri dish images can be enlarged and displayed on the GUI 112 so that the images can be easily switched by operating a mouse, a trackball, or a keyboard constituting the input / output terminal. To do.

  In the system of the present invention, shooting is performed under different optical conditions. The inspector cannot identify the exact bacterial species without confirming images under multiple optical conditions. Therefore, the GUI 112 can display images of a plurality of optical conditions. The inspector can display an image of any optical condition by operating the mouse, trackball, or keyboard of the input / output terminal, or display only a part of each petri dish and display multiple optical images. You may make it display the image of conditions simultaneously.

  Next, a fishing fungus candidate selection method by the fishing fungus candidate selection means 132 will be described with reference to FIG. A graph 1601 in FIG. 16A shows a feature vector distribution of colonies determined to be isolated in the feature space. The difference in the shape of the distribution points (square, circle, triangle) represents the classification result. Reference numeral 1604 denotes a classification identification boundary line. A solid line with a distribution point such as 1602 indicates a feature vector of an isolated colony that is a candidate for classification and a classification target, and a dotted line such as 1603 indicates a feature vector of a quasi-isolated colony that is handled only during classification. If there are a large number of isolated colonies sent as isolated colonies determined by the isolation determining means 130 and it is not necessary to catch all of them, the reliability of the classification result is high among these isolated colonies. Narrow things. A graph 1605 in FIG. 16B shows the final result of selecting a colony to be further selected from the isolated colonies sent from the isolation determination means, and the distribution points that are black are the final fishing. It is a fungus target. As the final target of the fungus, a colony near the center of the cluster as shown in 1606 is selected. This is because, if a colony near the classification identification boundary line such as 1607 is selected, since there are two or more classes of colonies, there is a high possibility that the classification result includes an erroneous colony. It is.

FIG. 17 shows an example of GUI 112 display. In 1701, a captured image, a classification result, and a fishing fungus candidate are displayed. In 1702, the user can select a mode to be corrected, and the corresponding result can be corrected. By clicking the selected mode and the image on 1701, it becomes possible to select the target to be corrected. For example, if you want to significantly correct the determination of isolated colonies or quasi-isolated colonies, check the isolated colony detection correction item in 1702, change the mode, and select a few colonies out of the colonies you want to check again, such as a mouse Is used to give correct judgment result information on the 1701 image. Isolation is determined again using the selected several colonies as correction information. In the isolation determination, it is assumed that an isolated colony class, a quasi-isolated colony class, and other classes are defined by the subspace already learned from the colony feature vector. Assume that the operator gives correction information on the determined colony to another class that the operator wants to determine as an isolated colony class. Assuming that the subspace of the isolated colony class already defined in learning is Z and the feature vector of the colony that is the target of this correction information is x, the subspace Z is a new one with Z '= (I + γxx ^t ) Z. Can be updated to subspace Z '. Here, I represents a unit matrix, and γ represents the degree of correction. After the subspace is updated with all the correction information given, classification is performed again. This correction method can be used not only for isolation determination but also for correction of colony classification, and it is possible to effectively correct the classification result with a small amount of correction information.

  Reference numeral 1703 in FIG. 17 is a portion for displaying the feature quantity vector as a two-dimensional distribution. The feature quantity on the vertical and horizontal axes is 1704 and can be arbitrarily selected by the user. In 1705, a histogram for one feature amount is displayed as an auxiliary function. Reference numeral 1706 denotes a slider that can change the weight of the feature amount. By selecting this slider, the distribution of 1703 can be changed and reflected in correction of the classification result.

  The outline of the weight change of the feature amount will be described with reference to FIG. In FIG. 18A, reference numeral 1801 denotes a distribution of feature quantity vectors without weighting. If two classes are classified for this distribution, a classification identification boundary line such as 1802 is obtained, but by changing the weighting of the feature value, for example, the weight is increased in the X-axis direction and the Y-axis is changed. On the other hand, when the weight is reduced, a distribution as 1803 in FIG. 18B is obtained, and a new classification result at the classification identification boundary line such as 1804 can be obtained. The feature weighting parameter may be set each time while the user looks at the GUI, or automatically weighted according to an optimum feature weighting parameter value set according to a parameter recipe described later. It doesn't matter.

  Note that the display of the GUI 112 shown in FIG. 17 is merely an example, and various changes can be made. For example, the image displayed in 1701 may be any one of a low-angle illumination image, a high-angle illumination image, and a transmission illumination image, and a plurality of these images may be displayed side by side. . Alternatively, the image before correction and the image reflecting the correction result may be displayed side by side, or the display of these images may be switched by pressing a display switching button. Further, not only the two-dimensional distribution but also the three-dimensional distribution may be displayed in 1703 for displaying the distribution of the feature vector. As a result of the classification, colonies classified into different groups may be displayed not only in different shapes such as squares, circles and triangles but also in different colors.

  Next, the extraction / classification parameter recipe will be described with reference to FIG. The secondary storage device 111 has a parameter recipe such as 1901 in FIG. 19, and determines the target of fishing bacteria, such as extraction parameters showing good results for medium types and specimens, classification feature weighting parameters, and classification algorithms. It has various parameters necessary for this as prior information. When the culture medium information of the petri dish is obtained from the petri dish information input unit 110, a plurality of appropriate parameter recipes are selected from all stored parameter recipes based on the obtained culture medium information of the petri dish. The results obtained by processing are sequentially displayed on the GUI 112 as in 1902. Since the user only has to confirm the result and select a result with good accuracy, the inspection speed can be reduced. In addition, even when correction is necessary, the correction time can be significantly shortened, and the settings can be reused by storing the correction result parameters in the secondary storage device 111. . In addition, when the optimal parameter recipe based on the culture medium information of the petri dish is established, the best one parameter recipe may be selected manually or automatically without selecting a plurality of parameter recipes. .

  As mentioned above, although the invention made by the present inventor has been specifically described based on the embodiment, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

The effects obtained by the representative inventions disclosed in this application will be briefly described as follows.
In other words, according to the present invention, it is possible to improve the classification accuracy from a plurality of parameter recipes compared to normal clustering, and the user can perform a high-accuracy operation by simply selecting a result from a plurality of classification results. Approach to the classification result is possible. In addition, when there are few isolated colonies to be fished on the petri dish, it is possible to stabilize the accuracy of the classification result by obtaining a feature data distribution sufficient to form clusters from other colonies.

101: Low-angle illumination unit, 102: High-angle illumination unit, 103: Imaging means, 104: Transmission illumination unit, 105: Shading plate, 106: Base, 107: Petri dish, 108: Optical system control unit, 109: Image processing unit, 110: Petri dish information input unit, 111: Secondary storage device, 112: GUI, 113: Petri dish transport means, 114: Stacker, 115: Petri dish transport means, 116: Stage control unit, 117: Fishing fungus, 118: Z stage 119: XY stage 120: Transmitted illumination unit 121 121 Upper illumination unit 122 122 Imaging means 123 123 Microplate 124 Saline 125 Petri dish 126 Image acquisition means 127 Colony extraction means 128: Distance between colonies measurement means, 129: Feature amount calculation means, 130: Isolation determination means, 131: Colony classification means, 132: Fishing fungus candidate selection means, 133: Fishing fungus target determination means

Claims (16)

Lighting means for illuminating bacterial colonies cultured on the culture medium stored in the container;
Imaging means for capturing a plurality of images by imaging the bacterial colonies illuminated by the illumination means;
Extracting a plurality of bacterial colonies from the plurality of images acquired by the imaging means, and based on the feature amounts respectively calculated from the extracted images of the plurality of bacterial colonies, Image processing means for performing classification and isolation determination;
Display means for displaying the plurality of bacterial colonies classified by the image processing means, and a bacterial colony of fishing fungus candidates selected from the plurality of bacterial colonies isolated by the image processing means;
A fishing fungus means for catching at least one of the bacterial colonies of the fungus candidate,
A bacterial colony fishing device characterized by comprising: A fishing device for bacterial colonies according to claim 1,
Furthermore, a storage means in which a plurality of parameter recipes are stored in advance,
Information input means for transmitting information containing at least medium information stored in the container to the image processing means;
Have
The image processing means uses the plurality of bacterial colonies using at least one of the plurality of parameter recipes selected based on information including at least medium information stored in the container transmitted by the information input means. A bacterial colony fishing device characterized by the above-mentioned classification. A fishing device for bacterial colonies according to claim 2,
The image processing means performs classification of the plurality of bacterial colonies using each of two or more parameter recipes selected from the plurality of parameter recipes stored in advance,
The bacteria colony fishing device, wherein the display means displays a plurality of results classified by each of the two or more selected parameter recipes. A fishing device for bacterial colonies according to claim 2 or 3,
The parameter recipe stored in the storage means includes at least one of a feature weighting parameter or a combination parameter of an image at the time of colony extraction as teaching information for each culture medium or specimen. Bacteria device. A fishing device for bacterial colonies according to claim 1,
The image processing means measures a distance between the extracted plurality of bacterial colonies, and based on the measured distance between the plurality of bacterial colonies and a feature amount respectively calculated from the images of the plurality of bacterial colonies. The bacterial colony fishing device characterized by performing isolation determination of the plurality of bacterial colonies. A fishing device for bacterial colonies according to claim 5,
The image processing means includes
Colony extracting means for extracting the plurality of bacterial colonies from the plurality of images;
An inter-colony distance measuring means for measuring a distance between the extracted bacterial colonies;
Feature amount calculating means for calculating feature amounts respectively calculated from the extracted images of the plurality of bacterial colonies;
Isolation determination means for performing isolation determination of the plurality of bacterial colonies based on the feature amount calculated from the images of the plurality of bacterial colonies and the measured distance between the plurality of bacterial colonies,
Colony classification means for classifying the plurality of bacterial colonies based on the feature amounts calculated from the images of the plurality of bacterial colonies,
Based on the bacterial colonies determined to be isolated and the classified result, a fishing fungus candidate selection means for selecting a fishing fungus candidate,
A bacterial colony fishing device characterized by comprising: A fishing device for bacterial colonies according to claim 6,
The isolation determination unit determines a quasi-isolation colony used for classification by the colony classification unit in addition to the isolation determination of the plurality of bacterial colonies. A bacterial colony fishing apparatus according to any one of claims 1 to 7,
An apparatus for fishing a bacterial colony, wherein the display means displays an item for correcting the displayed plurality of classified bacterial colonies or the selected bacterial colonies of fishing fungi candidates. A bacterial colony fishing apparatus according to any one of claims 1 to 8,
The illumination means transmits an upper illumination means for illuminating a bacterial colony cultured on the culture medium stored in the container from above, and transmits the bacterial colony cultured on the culture medium stored in the container through the container. And a illuminating device for transmitting and illuminating the bacteria colony fishing device. A bacterial colony fishing apparatus according to any one of claims 1 to 9,
The bacterial colony fishing apparatus characterized in that the feature amount is calculated using color information of colonies in at least two images photographed with illumination having different azimuth or elevation angles including transmission. Illuminating bacterial colonies cultured on a culture medium contained in a container;
Imaging the illuminated bacterial colonies to obtain a plurality of images;
Extracting a plurality of bacterial colonies from the acquired plurality of images, and classifying and isolating the extracted plurality of bacterial colonies based on feature quantities respectively calculated from the extracted plurality of bacterial colony images A step of making a determination;
Displaying the plurality of classified bacterial colonies and the isolated colony selected from the plurality of bacterial colonies isolated on a display means;
Fishing at least one of the displayed bacterial colonies of fishing fungus candidates;
A method for fishing bacterial colonies, comprising: A method for fishing bacterial colonies according to claim 11,
In the step of performing classification and isolation determination of the plurality of extracted bacterial colonies, the plurality of bacterial colonies are used by using each of a plurality of parameter recipes selected based on information including at least medium information stored in the container. Classification,
In the displaying step, a plurality of results classified by each of the selected plurality of parameter recipes are displayed. A method for fishing bacterial colonies according to claim 11,
In the step of performing classification and isolation determination of the plurality of extracted bacterial colonies, using one parameter recipe selected based on information including at least medium information stored in the container, the plurality of bacterial colonies A method for fishing bacterial colonies characterized by classifying. A method for fishing a bacterial colony according to claim 12 or 13,
A method for fishing a bacterial colony, wherein the parameter recipe includes at least one of a feature weighting parameter or an image combination parameter at the time of colony extraction as teaching information for each medium or specimen. A method for fishing bacterial colonies according to claim 11,
In the step of performing classification and isolation determination of the plurality of extracted bacterial colonies,
The distance between the plurality of extracted bacterial colonies is measured, and the plurality of bacteria is based on the calculated distance between the plurality of bacterial colonies and the feature amount calculated from the images of the plurality of bacterial colonies. A method for fishing bacterial colonies, comprising performing colony isolation determination. A method for fishing bacterial colonies according to claim 11,
Further, the step of correcting the result of the plurality of the bacterial colonies classified or the result of the bacterial colony of the fishing fungus candidate selected from the plurality of bacterial colonies isolated and displayed on the display means;
Re-displaying the modified results of the sorted plurality of bacterial colonies or the bacterial colonies of the selected fishing fungus candidates;
A method for fishing bacterial colonies, comprising: