JPH07200820A - Image recognition processing method - Google Patents

Abstract

PURPOSE:To provide an image recognition processing method which can automatically and accurately detect the number of tablets contained in each medicine bag. CONSTITUTION:In e system which detects the number of tablets 9 each contained in each transparent medicine beg 1.5, the beg 15 set at a checking position is photographed by a CCD camera 2. The image acquired by the camera 2 is sent to an image recognition processor 3 which counts the number of tablets 9. In this image recognition processing, a contact part between the images of two tablets 9 is detected and both images are separated from each other at this detected contact pert. Thus, the images of tablets 9 contained in the beg 15 are separated from each other. Then, the number of connection components included in these separated images of tablets 9 are counted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition processing method for detecting the number of objects based on original images obtained by photographing a plurality of objects.

[0002]

2. Description of the Related Art Conventionally, a tablet packing apparatus (1) as shown in FIG. 1 has been developed in order to automate tablet packing operations in hospitals and pharmacies. The tablet packaging device (1) includes a tablet supply unit (11) in which a plurality of types of tablets are stored according to type,
Various tablets (9) discharged from the tablet supply unit (11)
Automatically into the transparent packing paper (14) via the belt conveyors (12) (13), and the packing mechanism (17) seals the packing paper (14) for each person. Is. In addition, a printing mechanism (not shown) for automatically printing the patient name, usage, etc. on the surface of the packaging paper (14) is provided in the transport path of the packaging paper (14). By separating the packaging paper (14) thus obtained at the sealing portion for each person, the medicine bag (15) for each patient shown in FIG. 18 is obtained. The medicine bag (15) is provided with a printing section (16) for patient name, usage, etc.
In addition, since a necessary number of tablets (9) of a plurality of types to be taken at once by the patient are enclosed therein, it is convenient and there is no risk of mistakes in taking.

[0003]

However, in the conventional packaging work using the above-mentioned tablet packaging apparatus, the inspector visually confirms that a plurality of tablets to be packaged are accurately enclosed in one medicine bag. However, there is a problem that the efficiency of the whole packaging work is lowered by this complicated inspection work.

By the way, widely in the field of image recognition,
Various methods have been proposed for extracting the contour shape and the characteristic part of each object based on the images of a plurality of target objects (for example, Japanese Patent Publication 1-30183 [G06F15-70], Japanese Patent Publication 2-4948 [G06F15-7].
0], Japanese Patent Publication No. 3-57501 [G06F15-70], etc.). However, in the inspection of the tablets (9) in the medicine bag (15) as shown in FIG. 18, the tablets are often overlapped with each other, and the overlapping states are also different. In such a case, in the detection of the number of tablets using the conventional image recognition method, there is a possibility that two tablets overlapping each other may be counted as one tablet,
It is difficult to detect the correct number. Therefore, it is an object of the present invention to provide an image recognition processing method capable of automatically and accurately detecting the number of tablets enclosed in a medicine bag.

[0005]

An image recognition processing method according to the present invention binarizes an original image obtained by photographing a plurality of objects in a first step. In the second step, a pair of contact points located at both ends of a region where two objects contact or intersect each other are detected in the binarized image. Next, in a third step, the pair of contact points are connected to each other to create a boundary line between the images of the two objects. Then, in a fourth step, the images of the plurality of objects are individually divided by separating the images of the two objects that are in contact with each other at the boundary. Then, in a fifth step, the number of connected components included in the image is counted.

In another image recognition processing method according to the present invention, in the first step, contrast enhancement is applied to the original image, and in the second step, the edge of the image is enhanced based on the contrast-enhanced image. To detect. Next, in the third step, the detected edges are thinned to extract only the effective thin lines, and in the fourth step, the images of the two objects contacting each other are separated by the effective thin lines. ,
Divide images of multiple objects individually. Then, in a fifth step, the number of connected components included in the image is counted.

[0007]

In the first image recognition processing method, when two objects are in contact with each other or in contact with each other in an overlapping manner, the contact points of the images of these objects are, for example, point a shown in FIG. Since a pair of contact points such as point b and point c and point d occur, the boundary line between two objects can be identified by connecting such a pair of contact points to each other. Therefore, by separating the images of the two objects at the boundary line, it is possible to individually separate the images of the plurality of objects.
Therefore, the number of objects can be detected by counting the number of connected components included in the image.

In the second image recognition processing method,
First, by subjecting an image to contrast conversion, the contours of a plurality of objects and their contact portions are emphasized with respect to other image portions. Then, by detecting the edges of the image based on the contrast-enhanced image,
The contour lines of a plurality of objects and the boundary lines between them are found. If thinning is applied to these edges, beards and noises are removed, and only effective thin lines can be extracted. Therefore,
By separating the images of two objects in contact with each other at the position of the effective thin line, the images of a plurality of objects in the medicine bag can be individually divided.

[0009]

According to the image recognition processing method of the present invention, the images of objects which are in contact with each other or overlap with each other are reliably separated, and the number of objects is accurately detected.

[0010]

EXAMPLES An example of implementing the present invention in a tablet inspection system will be described below. As shown in Fig. 1, a tablet packing device
(1) has the same structure as the conventional one. Tablet packaging equipment
The packaging paper (14) discharged from (1) is further transported in the horizontal direction, and a camera (2) for photographing the surface of the medicine bag (15) is arranged above the transport path. Further, a planar illuminator (21) for illuminating the back surface of the medicine bag (15) is arranged at a position below the transport path. The camera (2) is a monochrome CCD camera having 410,000 pixels.

The image signal obtained from the CCD camera (2) is sent to the image recognition processing device (3) composed of a microcomputer and the like, and the tablet (9) enclosed in the medicine bag (15).
It is used for counting the number of. Also, an image recognition processing device
A monitor (4) is connected to (3) to display the process of image processing.

FIG. 2 shows a series of image recognition processing procedures executed by the image recognition processing device (3). First, in step S1, the original image from the CCD camera (2) is captured.
FIG. 8 is an example of an original image captured by the image recognition processing device (3). In this example, nine tablets are enclosed in the medicine bag, and eight of them are in contact with each other. Therefore,
In the conventional method, it is difficult to count the number. In the original image in FIG. 8, some noise is mixed around the image portion of the tablet.

Next, in step S2 of FIG. 2, the original image is binarized at a predetermined threshold level. As a result, each pixel forming the original image is represented by "0" or "1". FIG. 9 shows a binary image in which "0" is white and "1" is black. In the binary image, the noise included in the original image in FIG. 8 remains around the image portion of the tablet.

Next, in step S3 of FIG. 2, noise removal processing consisting of contraction and expansion of the binary image (for example, "basic technique of image processing", published by Technical Review Co., 53-56).
(See page). The contraction / expansion process is a process of deleting or multiplying all the boundary pixels of the connected components in the image,
In this embodiment, after the contraction process is performed twice, the expansion process is performed twice. As a result, the printing on the surface of the medicine bag (15) is deleted from the image and the isolated points in the image are removed.
In the above example, the noise included in the original image of FIG. 9 is removed.

Subsequently, in step S4 of FIG. 2, a labeling process is performed on the binary image that has been subjected to the noise removal process (for example, "basic technique of image processing" published by Technical Review Co.
(See page). The labeling process is a process in which, when adjacent pixels are present in the image data, the image group is regarded as one group and a label is assigned, and adjacent pixels are present in one of upper, lower, left and right. Sometimes 1
There are 4-connected labeling that is regarded as a group, and 8-connected labeling that is regarded as 1 group when adjacent pixels are present in any of up, down, left, right, and diagonal directions. In tablet inspection, 4-connection labeling, which is excellent in terms of figure separation, is effective.

Then, in step S5 of FIG. 2, chain coding processing is performed on the connected component in the image to which the label is assigned (see, for example, "Basic Techniques of Image Processing" published by Technical Review Company, pages 76-78). Do. The chain coding process is a process of tracing the contour or line segment of the connected component and giving a direction index (chain code) of “0” to “7” in that direction. With respect to each of the images assigned to, the contour (outer contour) formed by the outer shape of the tablet and the contour (inner contour) of a gap formed between a plurality of tablets in contact with each other are tracked. And
Record the chain code for each label.

Next, for each connected component to which a label is assigned, the contact point of the image caused by the overlapping of tablets is detected based on the recorded chain code of the contour line. FIG. 3 shows a contact detection procedure. First, contact candidate points are selected in step S61. As for the contact point candidate, the chain code is 0 → 6, as shown in FIG.
1 → 7, 2 → 0, 3 → 1, 4 → 2, 5 → 3, 6 → 4, 7
In the case of a combination of chain codes that changes abruptly like → 5 and changes from 0 → 6, 0 → 7 → 6 or 7 →
The change point is also set when one or two chain codes are present between the chain codes such as 7 → 6. Therefore, there are a total of 24 combinations of chain codes that should be changed.

Next, in step S62 of FIG. 3, at the contact point candidate, the angle of change in direction around the candidate point is calculated based on the contour lines of the front and rear 5 pixels. For example, in FIG.
In the case of the contact point candidate point generated by the contact of two tablets as shown in (a), the angle θ of the direction change is an acute angle, but FIG.
As described above, the angle θ is an obtuse angle if it is a contact point candidate point caused by noise generated on the contour line of one tablet. Therefore, in step S63, the angle θ is set to a predetermined threshold value (for example, 1
20 degrees), if it is smaller than the threshold value, it is determined to be a true contact point in step S64, and if it is equal to or larger than the threshold value, it is determined to be noise on the contour line in step S65.

FIGS. 6 (a) and 6 (b) are enlarged views in the vicinity of the direction change points in FIGS. 5 (a) and 5 (b), and the circles represent the pixels forming the figure. Here, a chain code that changes from 5 to 3 is detected as a contact point candidate point, and the opening angle of the fifth pixel before and after the candidate point as the center is detected as the direction change angle. In this case, the contact point candidate of FIG. 6A is determined to be a true contact point, and the contact point candidate of FIG. 6B is determined to be noise on the contour line and ignored.

Regarding the true contact point due to the overlapping of tablets,
As shown in FIG. 7, a pair of contacts (a, b), which are paired with each other,
(c, d) exists. Therefore, in step S7 of FIG. 2, such contact pairs are created for all labeled connected components. To create a contact pair, for example, a method of finding two contacts that satisfy the following conditions 1 to 3 is adopted. (Condition 1) The angle θ ₁ between two contact points (for example, the contact points a and b in FIG. 7, the direction change vectors A and B for the contact points a and b in FIG. 7) between the two contact points to be judged as a pair θ ₁ ( 0 ° ≤ θ ₁ ≤ 180 °) is the predetermined threshold
(For example, 150 °) or more. (Condition 2) The contact point vector of interest and a straight line from that contact point to another contact point (for example, for contact points a and b in FIG. 7,
An angle θ ₂ formed by a straight line E from the contact point a to the contact point b is equal to or larger than a predetermined threshold value (for example, 150 °). (Condition 3) Among the contact pairs that satisfy the above conditions 1 and 2,
The distance between the two points is equal to or less than a predetermined threshold value (for example, in the case of a circular tablet, 0.3 to 1.0 times the diameter), and is the minimum. In the case of FIG. 7, the contacts a and b or c and d are determined to be a pair, but the contacts a and c or a and d are not determined to be a pair.

After the contact pairs have been created for all the labels, in step S8 of FIG. 2, the two contacts determined to be a pair are connected to each other by a straight line to create a boundary line. The attached graphic element is separated into the connected components of the two tablet graphics at the boundary line. This processing is performed by replacing the pixel "1" on the boundary with "0". FIG. 10 shows a state in which, in the example of the binary image of FIG. 9, contact pairs are created at the contact portions of two tablets that are in contact with each other, and these contact pairs are interconnected to create a boundary line. FIG. 11 shows a state in which the binary image is divided into images of individual tablets at the boundary line.

Then, in step S9 of FIG. 2, relabeling is performed on the divided binary image of the image of each tablet. Then, the number of tablets is recognized by counting the number of labels in step S10. In the example of FIG. 11, the number of tablets is counted as 9. Then, it is determined whether the recognized quantity of tablets matches the expected quantity, and if they do not match, the medicine bag is automatically discharged as defective, or the medicine bag is marked as defective.

According to the image recognition processing method described above, an accurate contact pair is created especially when the overlapping area of tablets is small, and as a result, the number of tablets can be recognized with extremely high accuracy.

FIG. 12 shows a processing method for efficiently separating the images of two tablets which overlap each other in the contraction processing of step S3 of FIG. That is, in addition to the deletion of the boundary pixel described above, when the binary state of 3 × 3 pixels shown in FIG. 12 has the illustrated pattern, the pixel at the position marked with a double circle is also set to “0” at the same time. As another processing method, when the pattern of FIG. 12 is displayed, the pixel at the position marked with a double circle is set to “0” only when the pixel at the position marked with a square is “1”. A method is also possible. This method
The purpose is to shrink the double circle pixels after determining whether or not the pattern of FIG. 12 is a pattern caused by truly overlapping tablets. By adding this determination, it is possible to prevent erroneous determination that the edge generated in the capsule lid portion of the capsule is the overlapping of the tablets.

Further, in the contraction processing, it is possible to add processing for removing isolated points. For example, when all 8 pixels in the vicinity of the pixel of interest are "1", the pixel of interest is set to "0". In addition to the above method, as shown in FIGS.
A method can be adopted in which when two pixels or eight neighboring pixels are all "1", a pixel surrounded by these pixels is "1", and when all are "0". The latter method basically aims to remove the isolated points of two or less connected pixels, but the isolated points as shown in FIG. 13C are judged to be meaningful pixels and are not removed.

The above-described image recognition processing method shown in FIG. 2 is particularly effective when the overlapping area of the tablets is small.
4 shows the procedure of the image recognition processing method effective when the overlapping area of the tablets is large. Therefore, when the contact pair detection fails in the method of FIG. 2, if the images are separated by the method of FIG. 14, it is possible to surely recognize the quantity.

First, in step S11, a CCD camera
A grayscale image of 512 pixels × 512 lines is captured from (2). Also in this case, as shown in FIG. 1, the illumination by the illuminator (21) is performed from below the medicine bag (15). This is because when the illumination from above the medicine bag (15) is adopted, the image information of the tablet is lost due to the reflection on the surface of the medicine bag, or in the case of tablets or capsules with knurls on the surface, knurls or capsules This is because there is a drawback that the step portion of is detected as an edge.

Next, contrast conversion is performed in step S12 without binarizing the grayscale image. The density conversion formula used in this case is well known (see, for example, the magazine "Electronic Life", February 1989, pages 145-153). This facilitates extraction of the overlapping portion of the tablet.

Subsequently, edge detection filter processing is performed in step S13 of FIG. In the examples, as a result of examining various filter processing methods, as the best method, processing using a sobel filter of the first derivative (see, for example, magazine "Electronic Life", December 1988, pages 148 to 155).
It was adopted. As a result, the outline of the tablet and the overlapping area between the tablets are emphasized.

Further, in step S14, the contour-enhanced image is binarized by a predetermined threshold level to extract only contours (edges).

Then, in step S15, thinning is performed. The thinning is a process of extracting a core line passing through the center of the widthwise edge image in the width direction.
Adopt ch algorithm (for example, Tamura “Various considerations on thinning method” IEICE Technical Report, 1975 PR
See L75-9966). As a result, the boundary line of the overlapping portion with the contour lines of the plurality of tablets is obtained as a line image.

Then, in step S16, whiskers and noise are removed from the line image. The conditions for removal are as follows. (1) 1-pixel isolated point (2) Lines that do not include branch points or intersections and have 20 or fewer connected pixels (3) The number of connected pixels between an end point and a branch point or intersection is 1
Lines less than 0

Further, in step S17, the effective thin line necessary for the difference extraction step (S18) with the next binary image is selected. This is a procedure for removing long whiskers and loop-like pseudo-rays that remain without being removed even in step S16 described above, and FIG. 15 shows a specific procedure thereof. First, in step S21, an inner contour and an outer contour are extracted based on the line image. Then step S2
At 2, inflate these contour lines inward and outward,
Create a search area. For example, in the case of a line image in which two circular tablets overlap as shown in FIG.
The heavy lines M and N are created, and the area between them is used as the search area. The line E outside the search area is excluded from the target and is removed.

Subsequently, in step S23 of FIG. 15, a branch point or an intersection existing in a fine line in the search area is detected. In the example of FIG. 16, a branch point exists at a position surrounded by a circle in the search area. After that, in step S24 of FIG. 15, the line protruding from the search area at a distance of 20 pixels or more from the branch point or the intersection is determined to be an effective thin line. In the example of FIG. 16, A and D in the figure are effective thin lines, and the other lines B and C are removed as ineffective thin lines. The line D is a thin line that should be invalidated originally, but it does not pose a problem in recognizing the number of tablets as described later.

After selecting the effective thin line, step S in FIG.
At 18, the difference between the separately created binary original image and the image of the effective thin line is extracted, thereby separating the images of the two tablets that are connected to each other. FIG. 17 shows the result of the difference extraction in the example of FIG. As shown, the images of the two tablets are separated by an effective thin line. Although the effect of the line D remaining without being removed appears in the binary image of FIG. 17, this does not cause any problem in recognizing the quantity.

After that, labeling is performed in step S19 of FIG. 14, and the number of labels is counted in step S20 to recognize the number of tablets. Then, it is determined whether or not the recognized quantity of tablets matches the expected quantity, and if they do not match, the medicine bag is automatically discharged as defective, or the medicine bag is marked as defective.

According to the above image recognition processing method, even when two tablets are largely overlapped with each other, the illumination light leaks from the vicinity of the overlapped portion and the boundary line is surely detected by the edge detection. Separation is ensured and accurate quantity recognition is realized. Further, by combining the method of FIG. 2 and the method of FIG. 14, it is possible to inspect various medicine bags in the overlapping state of tablets.

In addition to the image recognition processing method of FIG. 2 or 14, the peripheral length or area of the binary image of the divided tablet is calculated and compared with the data registered in advance for each type of tablet. As a result, the accuracy of quantity recognition can be further improved, or the type of tablet can be discriminated.

The above description of the embodiments is for explaining the present invention, and should not be construed as limiting the invention described in the claims or limiting the scope. Further, the configuration of each part of the present invention is not limited to the above embodiment, but various modifications can be made within the technical scope described in the claims. For example, it goes without saying that the present invention can be widely applied to not only the above-described tablet inspection of the tablet packaging device but also the number recognition of a plurality of objects.

[Brief description of drawings]

FIG. 1 is a perspective view showing a configuration of a tablet inspection system embodying the present invention.

FIG. 2 is a flowchart showing a procedure of an image recognition processing method.

FIG. 3 is a flowchart showing a procedure of contact detection.

FIG. 4 is a diagram illustrating a method of selecting contact point candidate points.

FIG. 5 is a diagram illustrating a method of discriminating between a true contact point and noise.

FIG. 6 is a diagram illustrating a method of calculating a direction change angle around a contact point.

FIG. 7 is a diagram illustrating a method of discriminating contact pairs.

FIG. 8 is a diagram showing original images of a plurality of tablets.

9 is a diagram showing a binary image in the example of FIG.

FIG. 10 is a diagram showing a state in which a contact pair is created in the above example.

FIG. 11 is a diagram showing a state in which images of a plurality of tablets are divided in the above example.

FIG. 12 is a diagram illustrating a method of improving the efficiency of figure separation in the contraction processing.

FIG. 13 is a diagram illustrating a method of improving the efficiency of isolated point removal in the contraction processing.

FIG. 14 is a flowchart showing the procedure of another image recognition processing method.

FIG. 15 is a flowchart showing a procedure for selecting an effective thin line.

FIG. 16 is a diagram illustrating a method of removing a beard or a pseudo line generated at a branch point or an intersection.

FIG. 17 is a diagram showing a binary image obtained by extracting a difference from an effective thin line.

FIG. 18 is a perspective view of a medicine bag in which a plurality of tablets are enclosed.

[Explanation of symbols]

 (1) Tablet packing device (11) Tablet supply unit (14) Packaging paper (15) Medicine bag (2) CCD camera (21) Lighting equipment (3) Image recognition processing device (4) Monitor (9) Tablet

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] March 4, 1994

[Procedure Amendment 1]

[Document name to be amended] Statement

[Name of item to be corrected] Claim 2

[Correction method] Change

[Correction content]

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0006

[Correction method] Change

[Correction content]

In another image recognition processing method according to the present invention, in the first step, contrast enhancement is applied to the original image, and in the second step, the edge of the image is enhanced based on the contrast-enhanced image. To detect. Next, in the third step, only the effective thin lines are extracted after thinning the detected edges, and in the fourth step, the images of the two objects contacting each other are separated by the effective thin lines. Thus, the images of the plurality of objects are individually divided. Then the fifth
In step, the number of connected components included in the image is counted.

[Procedure 3]

[Document name to be amended] Statement

[Correction target item name] 0008

[Correction method] Change

[Correction content]

In the second image recognition processing method,
First, by subjecting an image to contrast conversion, the contours of a plurality of objects and their contact portions are emphasized with respect to other image portions. Then, by detecting the edges of the image based on the contrast-enhanced image,
The contour lines of a plurality of objects and the boundary lines between them are found. After thinning these edges, remove beards and noises
Only fine lines are extracted. Use this to contact each other 2
By separating the images of one object at the position of the effective thin line, the images of a plurality of objects in the medicine bag can be individually divided.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0012

[Correction method] Change

[Correction content]

FIG. 2 shows a series of image recognition processing procedures executed by the image recognition processing device (3). First, in step S1, the original image from the CCD camera (2) is captured.
FIG. 8 is an example of an original image captured by the image recognition processing device (3). In this example, nine tablets are enclosed in the medicine bag, and eight of them are in contact with each other. Therefore,
In the conventional method, it is difficult to count the number. It should be noted that in the original image of FIG. 8, some noise due to the printing of the packaging paper is mixed around the image portion of the tablet.

[Procedure Amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0033

[Correction method] Change

[Correction content]

Further, in step S17, the effective thin line necessary for the difference extraction step (S18) with the next binary image is selected. This is a procedure for removing long whiskers and loop-like pseudo-rays that remain without being removed even in step S16 described above, and FIG. 15 shows a specific procedure thereof. First, in step S21, the original image is binarized.
The inner contour and the outer contour are extracted from the image . Next, in step S22, these contour lines are expanded inward and outward to create a search area. For example, in the case of a line image in which two circular tablets are overlapped as shown in FIG. 16, double lines M and N are created by enclosing a thin line L, and a space between them is set as a search area. The line E outside the search area is excluded from the target and is removed.

[Procedure correction 6]

[Document name to be corrected] Drawing

[Correction target item name] Fig. 16

[Correction method] Change

[Correction content]

FIG. 16

Claims (2)

[Claims] 1. Based on original images obtained by photographing a plurality of objects,
An image recognition processing method for detecting the number of objects, comprising: a first step of binarizing the original image; and a position of the binarized image at both ends of a region where two objects contact or intersect each other. The second step of detecting a pair of contact points, and the third step of connecting the pair of contact points to each other to create a boundary line between the images of two objects, and the image of the two objects contacting each other as a boundary line. Image recognition processing characterized by including a fourth step of individually dividing the images of a plurality of objects by separating the plurality of objects, and a fifth step of counting the number of connected components included in the image obtained thereby. Method. 2. Based on original images obtained by photographing a plurality of objects,
An image recognition processing method for detecting the number of objects, comprising: a first step of applying contrast enhancement to the original image; a second step of detecting an edge of the image based on the image with enhanced contrast; A third step of thinning out the effective edges to extract only effective thin lines; and a step of individually dividing the images of a plurality of objects by separating the images of two objects in contact with each other by the effective thin lines. An image recognition processing method comprising: four steps; and a fifth step of counting the number of connected components included in the image obtained thereby.