JP2015052972A - Measuring computer program, measuring device, and measuring method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a computer program capable of identifying a quantity measuring target region of an object in a container from a plurality of images obtained by imaging the container in mutually different directions.SOLUTION: A computer program causes a computer to execute processes of: detecting at least a part of a profile of an upper edge of a sidewall and at least a part of a profile of a bottom surface from each of a plurality of images generated by imaging a container that has the bottom surface and the sidewall rising from the bottom surface in mutually different directions; generating a plurality of corrected images by performing an image correction process on each image so that a closed shape formed by the upper edge of the sidewall on the image or a shape of the bottom surface is identical to a shape viewed in a predetermined direction for each image; extracting a region in which a region surrounded by the profile of the upper edge of the sidewall overlaps a region surrounded by the bottom surface as a partial region of a content of the container for each corrected image; and identifying a region synthesized by aligning the partial regions of their respective corrected images as a quantity measuring target region of an object in the container.

Description

  The present invention relates to a computer program, a measuring apparatus, and a measuring method for measuring the amount of an object shown in an image.

  In the microbiological examination, microorganisms are cultured in a container such as a petri dish, and the number of colonies of the microorganisms is measured. However, measurement of the number of colonies places a heavy burden on the inspector, and the number of colonies may not be accurately counted. Thus, a contrast chamber has been proposed that allows the number of colonies to be automatically counted by a processor (see, for example, Patent Document 1).

  However, in order to use a dedicated device for colony measurement, such as the contrast chamber described in Patent Document 1, it is required that an inspector becomes familiar with the operation of the device. The contrast chamber described in Patent Document 1 is relatively expensive because it has a light source for properly illuminating the container, a read head for measuring colonies in the container, a mechanism for driving the read head, and the like. . Therefore, it is preferable that the number of colonies can be measured without using dedicated equipment such as the contrast chamber described in Patent Document 1.

  In addition, a technique for combining a plurality of captured images obtained by capturing a subject a plurality of times from different directions has been proposed (see, for example, Patent Document 2).

JP 7-500505 gazette JP 2006-287504 A

  In order to count the number of colonies of microorganisms in the petri dish using the technique disclosed in Patent Document 2 instead of using the petri dish equipment, the region in the petri dish is specified from each image taken of the petri dish. Is required.

  In one aspect, an object of the present invention is to provide a computer program capable of specifying a target region for measuring the amount of an object in a container from a plurality of images obtained by photographing the container from a plurality of different directions. And

  According to one embodiment, a computer program is provided. The computer program stores at least a part of the contour of the upper edge of the side wall and the contour of the bottom surface from each of a plurality of images generated by photographing the container having the bottom surface and the side wall rising from the bottom surface from different directions. Detects at least a part of the image, and for each of the plurality of images, the image is corrected so that the closed shape or the shape of the bottom surface formed by the upper edge of the side wall on the image becomes a shape when viewed from a predetermined direction. A process is performed to generate a plurality of corrected images, and for each of the generated corrected images, an area surrounded by the outline of the upper edge of the side surface and an area surrounded by the outline of the bottom surface overlap each other. Extracted as a partial area corresponding to a part of the inside of the container, and aligned the respective partial areas of the plurality of corrected images, combined, and measured the amount of objects in the container. Comprising instructions for executing the processing of identifying the computer as an area of interest to be.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

  A target region for measuring the amount of an object in the container can be specified from a plurality of images obtained by photographing the container from a plurality of different directions.

(A) is a figure which shows an example of the image of the petri dish which is an example of a container. (B) is a figure which shows the outline of the culture medium of the side wall vicinity of a petri dish. It is a schematic block diagram of the measuring device by one embodiment. It is a functional block diagram of a processing part. It is a figure which shows an example of the image in which the petri dish was reflected. It is a schematic diagram showing the relationship between each internal area | region of the image in which it was aligned, and the position of the detected colony. It is an operation | movement flowchart of a measurement process. (A) is a figure which shows an example of the edge detected from an image at the time of image | photographing a petri dish from diagonally upward. (B)-(e) is a figure which shows an example of the ellipse detected by carrying out the ellipse approximation of the set of edges from the image shown by (a), respectively. It is a figure which shows the table showing the relationship between the detected ellipse and the possibility of discrimination | determination of a top | upper surface and a bottom face.

Hereinafter, a measurement apparatus according to an embodiment will be described with reference to the drawings.
This measuring apparatus measures a predetermined amount of an object housed in a three-dimensional container such as a petri dish based on a plurality of images obtained by photographing the container from different directions with a camera.

  In the present embodiment, the container is a petri dish having a cylindrical shape and an upper end being an opening. In general, the petri dish is cylindrical and has a plate part having a bottom surface of a perfect circle and a side wall that rises almost vertically from the bottom surface, and a lid part that is substantially the same shape as the plate part and has a diameter larger than that of the plate part. It consists of a pair of two. The edge of the side wall of the dish part (the edge opposite to the bottom surface) is cut so as to be parallel to the bottom surface of the dish. In this embodiment, the object to be imaged is only the dish part from which the lid is removed, and in the following description, only the dish part of the petri dish may be pointed and described as a petri dish. An object to be measured is a colony of microorganisms cultured on a petri dish. The predetermined amount to be measured is the number of colonies. However, the container may not be a petri dish, and may be any container that has a top surface and a bottom surface, and the top surface is transparent or open. Strictly speaking, when the top surface of the container is an opening, there is no top surface, but the closed plane formed by the edge of the container side wall (the edge opposite to the bottom surface) corresponds to the top surface. Can be considered. In the following description, the closed plane formed by the edge of the side wall of the dish portion of the petri dish (the edge opposite to the bottom surface) may be referred to as the top surface. The object to be measured is not limited to a microbial colony. Furthermore, the predetermined amount is not limited to the number of colonies.

FIG. 1A shows an example of an image of a petri dish that is an example of a container, and FIG. 1B shows an outline of the culture medium in the vicinity of the side wall of the petri dish.
Unless the camera is arranged directly above the petri dish 110, the side wall 110a of the petri dish 110 hides a part of the bottom surface 110b of the petri dish 110 as shown in FIG. Does not show part of the bottom surface 110b. Further, as shown in FIG. 1B, in the vicinity of the side wall 110a of the petri dish 110, the culture medium 101 provided in the petri dish 110 rises along the side wall 110a. For this reason, even if the camera is arranged directly above the petri dish 110, the camera photographs the culture medium 101 near the side wall 110a from an oblique direction. Therefore, when there are a plurality of colonies 102 on the culture medium 101 in the vicinity of the side wall 110a, the plurality of colonies 102 appear to overlap with each other from the camera arranged directly above the petri dish 110, so that the colonies 102 also overlap on the image. Therefore, it is difficult to accurately count the number of colonies 102 based on the image.

  Therefore, the measuring apparatus according to the present embodiment detects the contours of the top and bottom surfaces of the petri dish on the images for each of a plurality of images obtained by photographing the petri dish from a plurality of different directions. . In addition, this measuring apparatus corrects each image so that the shape of the top surface or the bottom surface on the image becomes a shape when viewed from a predetermined direction. This measuring device is based on the positional relationship between the top surface and the bottom surface, and the region where the portion surrounded by the contour of the top surface overlaps the portion surrounded by the contour of the bottom surface is an area inside the petri dish where a colony may exist. It is obtained as an internal area corresponding to the part. And this measuring device aligns the internal area | region of each correction image. As a result, this measuring device identifies the region in the petri dish on each image by identifying the common region that overlaps between the images and the non-common portion that does not overlap among the internal regions of each image. To do. And this measuring device calculates | requires the number of the colonies contained in the internal area | region of each image so that the same colony may not be counted twice in a common part.

  FIG. 2 is a schematic configuration diagram of a measuring apparatus according to one embodiment. As shown in FIG. 2, the measuring device 1 includes an interface unit 11, a display unit 12, a storage unit 13, a processing unit 14, and a storage medium access device 15. The measuring device 1 may be, for example, a stand-alone computer or a server computer connected to a communication network. The measuring apparatus 1 measures the number of colonies of microorganisms in the petri dish based on the image obtained from the camera 2 and showing the petri dish.

  The camera 2 is, for example, a digital camera, and generates a plurality of images showing the same petri dish by photographing the petri dish from different directions. The camera 2 outputs the plurality of images to the measuring device 1. In addition, the camera which image | photographs a petri dish may not be 1 unit | set but multiple units | sets may be sufficient. The plurality of cameras may capture the petri dish from different directions, generate images in which the same petri dish is captured, and each camera may output the image to the measuring apparatus 1.

  The interface unit 11 includes, for example, a communication interface for connecting the measuring device 1 to the camera 2 and its control circuit. Such a communication interface can be an interface in accordance with a communication standard such as Universal Serial Bus (USB).

Alternatively, the interface unit 11 may include a communication interface for connecting to a communication network in accordance with a communication standard such as Ethernet (registered trademark) and a control circuit thereof. Then, the measuring apparatus 1 may acquire the plurality of images via a communication network from another device in which a plurality of images obtained by photographing the same petri dish from different directions with the camera 2 are stored. Good.
The interface unit 11 passes a plurality of images showing the petri dish to the processing unit 14.

  The display unit 12 includes a display device such as a liquid crystal display, for example. The display unit 12 is connected to the processing unit 14 via a video interface, for example, and displays the measurement result of the number of microbial colonies by the processing unit 14. The display unit 12 displays various information related to the application executed by the processing unit 14.

  The storage unit 13 includes, for example, a nonvolatile semiconductor memory and a volatile semiconductor memory. The storage unit 13 stores a measurement processing computer program executed by the processing unit 14 and various types of data used in the measurement processing. Further, the storage unit 13 may store an image received from the camera 2.

  The storage medium access device 15 accesses a storage medium 16 such as a magnetic disk, a semiconductor memory card, and an optical storage medium. Then, for example, the storage medium access device 15 reads a computer program for measurement executed on the processing unit 14 stored in the storage medium 16 and stores the computer program in the storage unit 13.

  The processing unit 14 includes one or a plurality of processors and their peripheral circuits. And the process part 14 performs the measurement process of the number of colonies of the microorganisms in a petri dish based on the several image obtained by image | photographing the same petri dish from a mutually different direction.

  FIG. 3 is a functional block diagram of the processing unit 14. The processing unit 14 includes a contour detection unit 21, a shape correction unit 22, an internal region extraction unit 23, a common part extraction unit 24, and a counting unit 25. Each of these units included in the processing unit 14 is, for example, a functional module realized by a computer program that operates on a processor included in the processing unit 14. Note that these units included in the processing unit 14 may be mounted on the measurement apparatus 1 as an integrated circuit separate from the processor included in the processing unit 14.

  The contour detection unit 21 detects at least part of the contour of the bottom surface of the petri dish and at least part of the contour of the top surface of the petri dish from each image.

FIG. 4 shows an example of an image showing a petri dish.
In the present embodiment, as described above, the petri dish 401 has a cylindrical shape with an open upper end. Therefore, the top surface outline 401 a and the bottom surface outline 401 b when the petri dish 401 is photographed obliquely from above are elliptical on the image 400. In general, since the side wall of the petri dish is low, in the image obtained by photographing the petri dish from an oblique direction, the top surface outline 401a of the petri dish overlaps with the bottom face outline 401b.

  Therefore, the contour detection unit 21 performs edge detection processing using an edge detection filter such as a sobel filter or a prewitt filter on each image, thereby obtaining an edge pixel that is a pixel in which an edge appears on the image. To detect. The contour detection unit 21 approximates a set of edge pixels with an elliptic equation using, for example, a least square method. The contour detection unit 21 detects two ellipses that overlap each other in the vertical direction on the image among the ellipses represented by the respective elliptic equations obtained by the approximation. The contour detection unit 21 sets the detected upper ellipse as the contour of the top of the petri dish, and the detected lower ellipse as the contour of the bottom of the petri dish.

  The contour detection unit 21 may detect the contour of the top surface of the petri dish and the contour of the bottom surface of the petri dish according to another contour detection method. For example, the contour detection unit 21 may detect the contour of the top of the petri dish on the image by template matching between the template representing the contour of the top of the petri dish and the edge image. The edge image can be, for example, a binary image in which the above edge pixels and pixels other than the edges have different values. Similarly, the contour detection unit 21 may detect the contour of the bottom surface of the petri dish on the image by template matching between the template representing the contour of the bottom surface of the petri dish and the edge image. The shape of the top surface and the shape of the bottom surface on the image changes depending on the angle between the normal to the bottom surface of the petri dish and the optical axis of the camera 2 and the distance between the petri dish and the camera 2. Therefore, it is preferable to prepare templates for the top and bottom surfaces of the petri dish for each assumed angle between the normal to the bottom surface of the petri dish and the optical axis of the camera 2 and for each assumed distance between the petri dish and the camera 2. The template is stored in advance in the storage unit 13, for example.

  The contour detection unit 21 notifies the shape correction unit 22 and the internal region extraction unit 23 of the elliptic equation representing the contour of the top surface of the petri dish and the elliptic equation representing the contour of the bottom surface of each image.

  The shape correcting unit 22 corrects each image so that the shape of the bottom surface or the top surface of the petri dish becomes a shape when the bottom surface or the top surface is viewed from a predetermined direction. In the present embodiment, the shape correcting unit 22 corrects each image so as to have a top or bottom shape when the petri dish is viewed from directly above or below the petri dish, that is, a perfect circle. For example, in the elliptic equation representing the contour of the bottom of the petri dish, the radius in the major axis direction is a and the radius in the minor axis direction is b. At this time, the shape correcting unit 22 uses the center of the ellipse representing the bottom as the origin, and for each pixel of the image, the distance from the origin to the pixel is multiplied by r / a along the major axis direction, and the minor axis direction The image is corrected so as to be multiplied by r / b along. Thereby, the contour of the bottom surface on each image after correction is represented by an equation of a circle with a radius r centering on the same position as the center of the contour of the bottom surface in the image before correction. Similarly, the contour of the top of the petri dish is also expressed by a circle equation with a radius r. However, the position of the center of the contour of the top surface in the image after correction is calculated by multiplying the vector from the center of the bottom surface to the center of the top surface in the image before correction by r / a in the major axis direction of the bottom surface. According to the vector multiplied by r / b in the direction, the position is away from the center of the bottom surface on the corrected image. The radius r is set to, for example, 100 to 300 pixels so that a colony can be detected in each corrected image.

  In the present embodiment, the outline of the bottom surface of the petri dish is similar to the outline of the top surface. Therefore, instead of the elliptic equation representing the contour of the bottom surface of the petri dish, the shape correcting unit 22 determines the radius a ′ in the major axis direction and the radius b ′ in the minor axis direction of the elliptic equation representing the contour of the top surface of the petri dish, Instead of the radius a in the major axis direction and the radius b in the minor axis direction of the elliptic equation representing the contour, it may be used.

  Or the shape correction | amendment part 22 performs a viewpoint conversion process with respect to each image, and the shape of the bottom face or top face of a petri dish becomes a shape when the bottom face or the top face is seen from a predetermined direction. In this way, each image may be corrected. For example, the shape correcting unit 22 is based on the major axis direction and the minor axis direction on the image, the ratio of the major axis direction radius to the minor axis direction radius, the major axis direction radius, and the angle of view of the camera 2. The positional relationship between the petri dish and the camera 2 can be estimated. Therefore, the shape correction unit 22 performs viewpoint conversion processing based on the positional relationship so that the image is obtained when the camera 2 captures the petri dish at a position where the optical axis of the camera 2 is parallel to the normal of the bottom surface of the petri dish. Then, each image is corrected so that the shape of the bottom surface of the petri dish and the shape of the top surface become a perfect circle.

  The shape correcting unit 22 extracts the corrected image, the circle equation corresponding to the contour of the top surface and the circle equation corresponding to the contour of the bottom surface of each corrected image, and the internal region extracting unit 23 and the common part extracting unit. Pass to part 24. Hereinafter, for convenience of explanation, the corrected image is simply referred to as a corrected image.

  The internal region extraction unit 23 extracts an internal region that is a region corresponding to a part of the interior of the petri dish from each corrected image.

  Referring to FIG. 4 again, a portion 402 where an area surrounded by the contour 401a of the top surface of the petri dish and an area surrounded by the contour 401b of the bottom surface overlaps the inner area, that is, the petri dish from the camera that has captured the image. Since it is an area where the inside culture medium can be seen directly, it can be seen that it is an area to be counted. In addition, the part contained only in either one of the area | region enclosed by the outline 401a and the area | region enclosed by the outline 401b respond | corresponds to the side wall of a petri dish. Since the positional relationship between the top and bottom contours does not change before and after image correction, the region surrounded by the top contour overlaps the region surrounded by the bottom contour on the corrected image. The part is the inner area.

  Therefore, the internal region extraction unit 23 refers to the circle equation representing the contour of the top surface and the circle equation representing the contour of the bottom surface in each correction image, and determines the region surrounded by the top surface contour and the contour of the bottom surface. A portion that overlaps the enclosed area is extracted as an internal area. Then, the internal area extraction unit 23 notifies the common part extraction unit 24 of information indicating the internal area of the correction image for each correction image. Note that the information representing the internal area can be, for example, a binary image having the same size as the size of the corrected image and having different pixel values and other pixel values included in the internal area.

  The common part extraction unit 24 aligns the corrected images. Thereby, the measuring device 1 can specify the area | region corresponded in a petri dish from the several image which image | photographed the same petri dish. Moreover, the common part extraction part 24 extracts the part which the internal area | region of two or more correction images overlaps among the internal area | regions of each correction image as a common part.

  For example, the common part extraction unit 24 extracts a plurality of feature points in the internal region for each correction image in order to use the correction images as reference points for aligning the correction images. For example, the common part extraction unit 24 performs a filtering process using a feature point extraction filter, such as a corner detection filter, on the internal region of each corrected image to extract feature points. Alternatively, the common part extraction unit 24 calculates a normalized cross-correlation value by performing template matching between a predetermined template and an internal region of each corrected image, and the normalized cross-correlation value is equal to or greater than a predetermined threshold value. A position in the internal region may be extracted as a feature point. For example, the template has a typical shape or luminance distribution of a colony. Alternatively, the template may represent a characteristic structure provided on the bottom surface of the petri dish.

  Next, the common part extraction unit 24 uses, for example, one of the corrected images as a reference image. Then, the common part extraction unit 24 sets any correction image other than the reference image as the target image. For example, the common part extraction unit 24 selects one of the feature points of the reference image as the first reference feature point, and selects one of the feature points of the target image as the second reference feature point. Then, the common part extraction unit 24 translates the target image so that the second reference feature point matches the first reference feature point. Thereafter, the common part extraction unit 24 calculates the number of feature points that match between the two corrected images while rotating the target image. The common part extraction unit 24 repeats the above processing while changing the combination of the first reference feature point and the second reference feature point, and the number of matching feature points, the amount of parallel movement of the target image at that time, and Find the rotation angle. Then, the common part extraction unit 24 affine-transforms the target image according to the parallel movement amount and the rotation angle when the number of matching feature points reaches the maximum value. As a result, the common part extraction unit 24 aligns the target image with the reference image.

  Alternatively, the common part extraction unit 24 may detect the position of the target image that most closely matches the reference image and the target image by performing pattern matching while changing the relative position between the reference image and the target image. And the common part extraction part 24 may align a target image with a reference | standard image by moving a target image to the position which most matches.

  The common part extraction unit 24 aligns all the images by repeating the above processing while changing the target image. The common part extraction unit 24 may use the aligned target image as a reference image for the next target image. Then, the common part extraction unit 24 examines which correction image includes the internal region for each pixel in a state where all the correction images are aligned. The common part extraction unit 24 associates, for each pixel, the identification number of the corrected image in which the pixel is included in the internal region. Pixels associated with the identification numbers of a plurality of corrected images are included in the common part of the plurality of corrected images.

  The common part extraction unit 24 notifies the counting unit 25 of the identification number of the corrected image associated with each pixel.

  The counting unit 25 measures a predetermined amount of the object in the container from each of the internal regions of each correction image so that the object included in the common part is not obtained redundantly for two or more correction images. In the present embodiment, the counting unit 25 calculates the number of colonies of microorganisms included in the common part from each of the internal regions of each correction image so that the number of colonies of microorganisms included in the common part is not duplicated. Obtain the number of microbial colonies.

  First, the counting unit 25 detects colonies from the internal area of each correction image. For this purpose, the counting unit 25 extracts, for example, a point that becomes a local peak where the luminance is equal to or higher than a predetermined value in the internal region. The counting unit 25 obtains a combination of peak points that are closer than a predetermined threshold among the extracted peak points, and obtains a luminance change of a pixel on a straight line connecting the peak points for each combination. If the luminance change is less than a predetermined threshold, the counting unit 25 detects the peak point included in the combination as one colony, and sets the center of each peak point included in the combination as the colony position. On the other hand, if the luminance change is equal to or greater than a predetermined threshold, the counting unit 25 detects each peak point included in the combination as a separate colony.

  Alternatively, for example, the counting unit 25 performs template matching while changing the relative position between the internal region of each corrected image and the template representing the colony, and calculates a normalized cross-correlation value for each pixel in the internal region. calculate. Then, the counting unit 25 determines that a colony exists in a pixel whose normalized cross-correlation value is equal to or greater than a predetermined detection threshold. Alternatively, the counting unit 25 may divide the internal region into a plurality of blocks and obtain a feature amount related to luminance, color, etc. for each block. And the counting part 25 inputs the feature-value into the discriminator learned beforehand so that it may output the determination result whether a colony exists, such as a support vector machine and an adaBoost discriminator, for every block. You may determine whether a colony exists.

  When a colony is detected from the internal area of each correction image, the counting unit 25 calculates the total number of detected colonies. At that time, the counting unit 25 determines whether or not the pixel determined to have a colony in each corrected image is included in the common part. If the pixel determined to have a colony is not included in the common part, that is, if it is a colony that appears only in one of the corrected images, the counting unit 25 increments the number of colonies by one.

  On the other hand, the counting unit 25 classifies the pixels in which the colonies determined to be included in the common part are detected for each combination of correction images overlapping with each other. For each combination of correction images, the counting unit 25 reflects the maximum value of the number of colonies detected from the common portion among the correction images included in the combination in the common portion between the correction images of the combination. The number of colonies.

  An example of colony count measurement is shown with reference to FIG. FIG. 5 is a schematic diagram showing the relationship between the internal areas of the three aligned corrected images and the position of the detected colony. In this example, the internal areas 501 to 503 of each corrected image are aligned. Among these, in the common part 511, the internal region 501 and the internal region 502 overlap. In the common portion 512, the internal region 501 and the internal region 503 overlap. Further, in the common portion 513, the internal region 502 and the internal region 503 overlap. In the common portion 514, all of the internal region 501 to the internal region 503 overlap.

  In FIG. 5, the colony 521 detected from the internal region 501 is represented by a circle. Similarly, the colony 522 detected from the internal region 502 is represented by a rhombus. A colony 523 detected from the internal region 503 is represented by an asterisk. In this example, two, two, and one colony are detected from the portions of the internal regions 501 to 503 that do not overlap with other internal regions, respectively. In the common portion 511 where the internal region 501 and the internal region 502 overlap, two colonies are detected from the internal region 501, while three colonies are detected from the internal region 502. Therefore, the counting unit 25 sets the number of colonies of the common portion 511 to 3.

Similarly, in the common portion 512 where the internal region 501 and the internal region 503 overlap, one colony is detected from the internal region 501, while two colonies are detected from the internal region 503. Therefore, the counting unit 25 sets the number of colonies of the common portion 512 to 2. Furthermore, in the common part 513 where the internal region 502 and the internal region 503 overlap, two colonies are detected from the internal region 502, while two colonies are also detected from the internal region 503. Therefore, the counting unit 25 sets the number of colonies of the common portion 513 to 2. Furthermore, in the common part 514 where all of the inner regions 501 to 503 are overlapped, three, one, and two colonies are detected from the inner regions 501 to 503, respectively. Therefore, the counting unit 25 sets the number of colonies of the common portion 514 to 3.
From the above, the counting unit 25 calculates the total number of colonies as 2 + 2 + 1 (the number of single regions) +3 (common portion 511) +2 (common portion 512) +2 (common portion 513) +3 (common) The portion 514) = 15.
As described above, the counting unit 25 can suppress colony detection omission by obtaining the number of colonies in the internal region and the common portion.

  Note that the counting unit 25 may use the number of colonies included in the common part as an average value of the number of colonies detected from the internal area of each correction image overlapping the common part. Alternatively, the counting unit 25 sets the number of colonies included in the common part to the number of colonies detected from the internal area of each corrected image overlapping the common part, and the photographing direction of the image before correction corresponding to the internal area It is good also as the sum total of the value which multiplied the weighting coefficient according to. In this case, as the position of the camera 2 at the time of shooting is closer to the top of the petri dish, that is, the smaller the angle between the optical axis of the camera 2 and the normal line of the bottom of the petri dish, The possibility that colonies appear to overlap each other is considered to be small. Therefore, it is preferable that the weighting factor increases as the angle formed by the normal line between the optical axis of the camera 2 and the bottom surface of the petri dish decreases. Further, the smaller the angle formed between the normal line of the bottom surface of the petri dish and the optical axis of the camera 2, the closer the shape of the bottom surface of the petri dish on the image is to a perfect circle. Therefore, the counting unit 25 calculates the ratio (b / a) of the radius b of the minor axis to the radius a of the major axis of the elliptic equation representing the bottom surface of the petri dish on the image before correction corresponding to each internal region overlapping with the common part. A value obtained by dividing the ratio (b / a) of each image by the total Σ (b / a) may be used as the weighting coefficient. In this way, by calculating the weighted average of the number of colonies according to the direction of the camera, the counting unit 25 can accurately calculate the number of colonies included in the common part.

Alternatively, when the position of the colony detected from one of the internal areas overlapping with the common part matches the position of the colony detected from the other internal area, the counting unit 25 assumes that those colonies are the same. The colony number may be incremented by one. On the other hand, when the positions of the colonies detected from each of the plurality of internal regions overlapping the common part are different from each other, the counting unit 25 counts the number of colonies as the detected number, assuming that those colonies are separate. May be. For example, in FIG. 5, in the common portion 513, the colony 531 is detected at the same position from both the internal region 502 and the internal region 503, so the counting unit 25 counts the colony 531 as one. On the other hand, the colony 532 is detected from the inner area 502 but not detected from the inner area 503. Conversely, the colony 533 is detected from the inner region 503 but not from the inner region 502. Therefore, the counting unit 25 counts the colonies 532 and 533 as separate colonies. As a result, the counting unit 25 sets the number of colonies included in the common portion 513 to three.
According to this modification, all the colonies detected from any one of the images are counted, and therefore the counting unit 25 can suppress estimating the number of colonies less than the original number of colonies.

  The counting unit 25 outputs the total number of detected colonies to the display unit 12. Alternatively, the counting unit 25 may store the total number of detected colonies in the storage unit 13.

FIG. 6 is an operation flowchart of the measurement process.
The contour detection unit 21 detects at least part of the contour of the top surface of the petri dish and at least part of the contour of the bottom surface from each of a plurality of images obtained by photographing the same petri dish from different directions (step S101). The shape correcting unit 22 corrects each image so that the shape of the bottom surface or top surface of the petri dish becomes a shape when the bottom surface or top surface is viewed from directly above the petri dish, that is, a perfect circle (step S102). ).

The internal area extraction unit 23 extracts, as an internal area, an area where the area surrounded by the outline of the bottom surface and the area surrounded by the outline of the top surface overlap each correction image (step S103). Then, the common part extraction unit 24 aligns the internal areas of the respective correction images, and extracts the common part where the internal areas of the plurality of correction images overlap each other, thereby specifying the area in the petri dish (step S104).
The counting unit 25 uses the method described with reference to FIG. 5 so that the number of colonies of microorganisms included in the common part is not redundantly obtained for correction images of two or more. Based on the number of colonies detected from each of the regions, the number of microbial colonies in the petri dish is obtained (step S105). And the measuring device 1 displays the number of colonies on the display part 12, for example. Thereafter, the measuring device 1 ends the measurement process.

  As described above, this measuring device includes an area in which the top surface and the bottom surface of the container overlap each other from each of a plurality of images obtained by photographing the container containing the object to be counted from different directions. Extract as a region. And this measuring device aligns each internal area, specifies a common part where a plurality of internal areas overlap, and then avoids double counting a predetermined amount of an object included in the common part. The predetermined amount of the object in the container is obtained by counting the predetermined amount of the contained object. Therefore, this measuring apparatus can specify a region to be measured from a plurality of images obtained by photographing the container from different directions without using dedicated equipment for container photographing. In addition, a predetermined amount of an object in the container can be obtained using such an image. In addition, since this measuring device can use an image obtained by photographing the container from an oblique direction, even when the object to be detected is present in the vertical direction in the vicinity of the side wall of the container, the predetermined amount of the object is accurately determined. Can count.

  In addition, when detecting the outline of the top surface and the bottom surface of the petri dish that is an example of the container by the ellipse approximation, the contour detection unit 21 detects an ellipse along a part of the top surface contour and a part of the bottom surface contour. It may be detected.

  FIG. 7A is a diagram illustrating an example of a set of edges detected from an image when a petri dish is photographed from diagonally above. FIGS. 7B to 7E are diagrams showing examples of ellipses detected by elliptically approximating a set of edges from the image shown in FIG. 7A.

  As shown in FIG. 7A, in the image 700, a set of edges 701 corresponding to the contour of the top surface and a set of edges 702 corresponding to the contour of the bottom surface overlap each other along the vertical direction. . Here, when an edge set of the image 700 is approximated by an ellipse, an ellipse 711 representing the contour of the top surface and an ellipse 712 representing the contour of the bottom surface shown in FIGS. 7B and 7C may be obtained. preferable. However, actually, an ellipse 713 that approximates the upper edge on the bottom contour image and the lower edge on the top contour image shown in FIG. 7D, or FIG. An ellipse 714 approximating the upper edge on the top contour image and the lower edge on the bottom contour image shown in e) may be obtained. An ellipse 713 represents the outline of the inner region, and an ellipse 714 represents the outline of the petri dish.

  By ellipse approximation of a set of edges, there is a high possibility that one or more of these four types of ellipses will be detected. Therefore, the contour detection unit 21 may determine the contour of the top surface and the contour of the bottom surface based on the number of ellipses detected and the positional relationship.

FIG. 8 is a diagram showing a table representing the relationship between the detected ellipse and whether or not the top and bottom surfaces can be distinguished.
In the table 800, each row corresponds to one combination of detected ellipses. Each column of the leftmost column 801 shows an identification number of a combination of detected ellipses attached for convenience. Each column in columns 802 to 805 represents a detected one of the ellipses corresponding to the contour of the top surface, the contour of the bottom surface, the contour of the internal area, and the contour of the petri dish with circles. A column 806 indicates an ellipse corresponding to the contour of the top surface and an ellipse corresponding to the contour of the bottom surface when the contour detection unit 21 can identify the contour of the top surface and the contour of the bottom surface. On the other hand, a column 807 indicates whether or not the contour detection unit 21 can identify at least one of the top surface contour and the bottom surface contour.

  As shown in the table 800, when two or more ellipses including two ellipses intersecting each other are detected, the contour detection unit 21 sets the upper side on the image of the two intersecting ellipses. The ellipse can be identified from the contour of the top surface, and the lower ellipse can be identified from the contour of the bottom surface. In addition, when three ellipses are detected and none of the combinations of the two ellipses intersect with each other, an ellipse corresponding to the contour of the inner region and the petri dish and the contour of the top surface or the bottom surface An ellipse corresponding to either one of these has been detected. In this case, the outermost ellipse corresponds to the outline of the petri dish, and the innermost ellipse corresponds to the outline of the inner region. If the upper side touches an ellipse corresponding to the outline of the petri dish and the lower side touches an ellipse corresponding to the outline of the internal region on the image of an ellipse other than these two (hereinafter referred to as the target ellipse for convenience), The ellipse corresponds to the contour of the top surface. On the other hand, when the upper side touches an ellipse corresponding to the outline of the internal region on the image of the target ellipse and the lower side touches an ellipse corresponding to the outline of the petri dish, the target ellipse corresponds to the outline of the bottom surface.

  If the detected combination of ellipses is other than the above, the contour detection unit 21 cannot identify the top surface contour and the bottom surface contour.

  However, when two ellipses are detected and one of the ellipses includes the whole of the other ellipse, the outer ellipse corresponds to the outline of the petri dish, and the inner ellipse corresponds to the inner region. In this case, the contour detection unit 21 re-ellipses the edge corresponding to the upper half on the inner ellipse image and the edge corresponding to the lower half on the outer ellipse image to obtain the contour of the bottom surface. The corresponding ellipse can be detected.

  When the contour detection unit 21 can identify at least one of the contour of the top surface and the contour of the bottom surface and the internal region based on the ellipse detected from the image according to the table 800, the image is used for the subsequent processing of the measurement processing. Use. On the other hand, if neither the top surface contour nor the bottom surface contour can be identified based on the ellipse detected from the image, the contour detection unit 21 may discard the image without using it for the subsequent processing. .

  Further, according to the modification, the outer shape of the container is not cylindrical, and may have another shape, for example, a rectangular parallelepiped shape. In this case, the contour detection unit 21 may approximate a set of edges according to the contour shape of the top surface and the contour shape of the bottom surface of the container. For example, if the outer shape of the container is a rectangular parallelepiped, the contour detection unit 21 may approximate the set of edges with a trapezoid. The shape correcting unit 22 may correct the image so that the trapezoid corresponding to the contour of the top surface and the trapezoid corresponding to the contour of the bottom surface are rectangular.

  According to still another modification, the common part extraction unit 24 aligns the effective partial areas detected from the respective images, and values of the pixels at the same position (luminance values or values of each color of red, green, and blue) ) May be averaged to generate a composite image. And the counting part 25 may detect a colony from the internal area | region in the synthesized image, and may obtain | require the sum total of the number of colonies.

  According to still another modification, the counting unit 25 may detect a pixel in which an object to be detected appears from the internal area of each image before correction. For example, when the objects to be detected are arranged in an oblique direction with respect to the bottom surface of the container, such as a culture colony near the side wall of the petri dish, the image before the correction is more on the image than the correction image. It is easy to separate the objects. In such a case, it is preferable that the counting unit 25 detects a pixel in which an object to be detected is captured from the internal region of the image before correction.

  In this case, the counting unit 25 can specify the position of the pixel where the object is detected on the corrected image by performing the same process as the process performed by the shape correcting unit 22 on the pixel where the object is detected. Further, the counting unit 25 moves the position of the pixel where the object is detected according to the parallel movement amount and the rotation amount of the image obtained at the time of alignment, so that the object on the corrected image after alignment is displayed. The position of the detected pixel can be specified.

  According to still another modification, the shape correction unit 22 is configured so that the shape of the top surface or bottom surface of the container is the shape when the top surface or bottom surface of the container is viewed from a direction other than directly above. The image may be corrected. In particular, when there are characteristic structures at a plurality of locations on the side wall of the container, the shape correction unit 22 detects any of the structures on the image by template matching, for example. Then, the shape correcting unit 22 can specify the photographing direction on the plane parallel to the bottom surface by obtaining the rotation angle from the detected position of the structure to the upper end or lower end of the contour of the top surface or the bottom surface of the container on the image. Further, as described above, if the container has a cylindrical shape, the shape correction unit 22 determines the ratio of the major axis length of the ellipse that approximates the contour of the bottom surface of the container to the length of the minor axis. The inclination of the shooting direction with respect to the normal can be specified. Then, the shape correction unit 22 may perform viewpoint conversion processing for each image so that the shooting direction on a plane parallel to the bottom surface of the container matches the inclination of the shooting direction with respect to the normal line of the bottom surface.

  According to still another modification, the counting unit 25 may obtain an amount related to the object other than the number of objects in the container, for example, the area of the object in the container, as the predetermined amount of the object in the container. . The object occupying the container can be, for example, a medium in a petri dish or a colony of microorganisms. In this case, the counting unit 25 may set the area of the object as the total number of pixels in which the object to be detected is shown in the internal regions of the aligned corrected images. However, also in this case, in the common part, the counting unit 25 determines the number of pixels in which the object is detected in the common part, for example, so that the number of pixels in which the object to be detected is reflected is not doubled. What is necessary is just to determine according to the method similar to said embodiment.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
From each of a plurality of images generated by photographing a container having a bottom surface and a side wall rising from the bottom surface from different directions, at least a part of the contour of the upper edge of the side wall and at least one of the contour of the bottom surface Part
For each of the plurality of images, image correction processing is executed so that the closed shape formed by the upper edge of the side wall on the image or the shape of the bottom surface is a shape when viewed from a predetermined direction. Generate multiple corrected images,
For each of the plurality of generated correction images, a region where the region surrounded by the outline of the upper edge of the side surface overlaps the region surrounded by the contour of the bottom surface corresponds to a part of the inside of the container. Extract as a partial area,
An area to be synthesized by aligning the partial areas of the plurality of correction images is specified as an area to be measured for the amount of the object in the container.
A computer program that causes a computer to execute processing.
(Appendix 2)
The amount of the object is the number of the object contained in the container;
Measuring the amount of the object is based on the number of the objects in the container based on the number of objects detected from each of the partial area and the partial area where the partial areas of the two or more corrected images overlap. The computer program according to attachment 1, wherein
(Appendix 3)
Measuring the amount of the object includes, in the common part, the maximum value of the number of objects detected from the part overlapping the common part in each of the partial regions of two or more images. The computer program according to attachment 2, wherein the number of objects is the number of objects.
(Appendix 4)
Measuring the amount of the object means that the number of objects detected from a portion overlapping each common portion of each of the partial regions of two or more images, and the shooting direction of the corresponding image is the container. The computer program according to appendix 2, wherein the number obtained by weighted averaging using a weighting factor that increases as it approaches the normal of the bottom surface is the number of objects included in the common part.
(Appendix 5)
Measuring the amount of the object includes aligning the two or more partial regions among the objects detected from a portion overlapping the common portion of each of the partial regions of the two or more images. When the two or more partial areas are aligned with each other, the objects that are in the same position are set as the same object. The computer program according to attachment 2, wherein the number of objects included in the common part is obtained as a separate object.
(Appendix 6)
The computer program according to any one of appendices 2 to 5, wherein the container is a petri dish, and the number of the objects is the number of colonies of microorganisms cultured in the container.
(Appendix 7)
The computer program according to any one of appendices 1 to 6, wherein the predetermined direction is a normal direction of a bottom surface of the container.
(Appendix 8)
The closed contour and the bottom contour formed by the upper edge of the side wall are circles,
Detecting at least a part of the contour of the closed shape and at least a part of the contour of the bottom surface is obtained by detecting an edge from the image for each of the plurality of images and approximating the set of edges to an ellipse. The computer program according to any one of appendices 1 to 7, wherein at least a part of the contour of the closed shape and at least a part of the contour of the bottom surface are detected.
(Appendix 9)
Detecting at least a part of the contour of the closed shape and at least a part of the contour of the bottom surface means that three or more ellipses are detected by ellipse approximation of the set of edges in any of the plurality of images. If two of the detected ellipses intersect, the upper ellipse on the image of the intersecting ellipses is defined as the closed shape outline, The computer program according to appendix 8, wherein the lower ellipse is defined as the contour of the bottom surface.
(Appendix 10)
From each of a plurality of images generated by photographing a container having a bottom surface and a side wall rising from the bottom surface from different directions, at least a part of the contour of the upper edge of the side wall and at least one of the contour of the bottom surface A contour detection unit for detecting a part;
For each of the plurality of images, image correction processing is executed so that the closed shape formed by the upper edge of the side wall on the image or the shape of the bottom surface is a shape when viewed from a predetermined direction. A shape correction unit that generates a plurality of corrected images;
For each of the plurality of generated correction images, a region where the region surrounded by the contour of the upper edge of the side wall and the region surrounded by the contour of the bottom surface corresponds to a part of the inside of the container. An internal region extraction unit for extracting as a partial region;
A common part extraction unit that identifies a region synthesized by aligning the partial regions of the plurality of correction images as a target region for measuring the amount of an object in the container;
Measuring device.
(Appendix 11)
From each of a plurality of images generated by photographing a container having a bottom surface and a side wall rising from the bottom surface from different directions, at least a part of the contour of the upper edge of the side wall and at least one of the contour of the bottom surface Part
For each of the plurality of images, image correction processing is executed so that the closed shape formed by the upper edge of the side wall on the image or the shape of the bottom surface is a shape when viewed from a predetermined direction. Generate multiple corrected images,
For each of the plurality of generated correction images, a region where the region surrounded by the contour of the upper edge of the side wall and the region surrounded by the contour of the bottom surface corresponds to a part of the inside of the container. Extract as a partial area,
An area to be synthesized by aligning the partial areas of the plurality of correction images is specified as an area to be measured for the amount of the object in the container.
A measuring method including that.

DESCRIPTION OF SYMBOLS 1 Measuring apparatus 2 Camera 11 Interface part 12 Display part 13 Storage part 14 Processing part 15 Storage medium access apparatus 16 Storage medium 21 Contour detection part 22 Shape correction part 23 Internal area extraction part 24 Common part extraction part 25 Counting part

Claims (7)

From each of a plurality of images generated by photographing a container having a bottom surface and a side wall rising from the bottom surface from different directions, at least a part of the contour of the upper edge of the side wall and at least one of the contour of the bottom surface Part
For each of the plurality of images, image correction processing is executed so that the closed shape formed by the upper edge of the side wall on the image or the shape of the bottom surface is a shape when viewed from a predetermined direction. Generate multiple corrected images,
For each of the plurality of generated correction images, a region where the region surrounded by the outline of the upper edge of the side surface overlaps the region surrounded by the contour of the bottom surface corresponds to a part of the inside of the container. Extract as a partial area,
An area to be synthesized by aligning the partial areas of the plurality of correction images is specified as an area to be measured for the amount of the object in the container.
A computer program that causes a computer to execute processing. The amount of the object is the number of the object contained in the container;
Measuring the amount of the object is based on the number of the objects in the container based on the number of objects detected from each of the partial area and the partial area where the partial areas of the two or more corrected images overlap. The computer program according to claim 1, wherein:   Measuring the amount of the object includes, in the common part, the maximum value of the number of objects detected from the part overlapping the common part in each of the partial regions of two or more images. The measurement computer program according to claim 2, wherein the computer program is the number of objects.   Measuring the amount of the object means that the number of objects detected from a portion overlapping each common portion of each of the partial regions of two or more images, and the shooting direction of the corresponding image is the container. The computer program according to claim 2, wherein a number obtained by performing weighted averaging using a weighting factor that increases as it approaches the normal line of the bottom surface is defined as the number of objects included in the common part.   Measuring the amount of the object includes aligning the two or more partial regions among the objects detected from a portion overlapping the common portion of each of the partial regions of the two or more images. When the two or more partial areas are aligned with each other, the objects that are in the same position are set as the same object. The computer program according to claim 2, wherein the number of objects included in the common part is obtained as a separate object. From each of a plurality of images generated by photographing a container having a bottom surface and a side wall rising from the bottom surface from different directions, at least a part of the contour of the upper edge of the side wall and at least one of the contour of the bottom surface A contour detection unit for detecting a part;
For each of the plurality of images, image correction processing is executed so that the closed shape formed by the upper edge of the side wall on the image or the shape of the bottom surface is a shape when viewed from a predetermined direction. A shape correction unit that generates a plurality of corrected images;
For each of the plurality of generated correction images, a region where the region surrounded by the contour of the upper edge of the side wall and the region surrounded by the contour of the bottom surface corresponds to a part of the inside of the container. An internal region extraction unit for extracting as a partial region;
A common part extraction unit that identifies a region synthesized by aligning the partial regions of the plurality of correction images as a target region for measuring the amount of an object in the container;
Measuring device. From each of a plurality of images generated by photographing a container having a bottom surface and a side wall rising from the bottom surface from different directions, at least a part of the contour of the upper edge of the side wall and at least one of the contour of the bottom surface Part
For each of the plurality of images, image correction processing is executed so that the closed shape formed by the upper edge of the side wall on the image or the shape of the bottom surface is a shape when viewed from a predetermined direction. Generate multiple corrected images,
For each of the plurality of generated correction images, a region where the region surrounded by the contour of the upper edge of the side wall and the region surrounded by the contour of the bottom surface corresponds to a part of the inside of the container. Extract as a partial area,
An area to be synthesized by aligning the partial areas of the plurality of correction images is specified as an area to be measured for the amount of the object in the container.
A measuring method including that.

Images