JP5163985B2 - Granular product type inspection device - Google Patents

Description

  The present invention relates to a granular material type inspection apparatus that automatically determines the type of granular article, and more particularly to a granular material type inspection apparatus that is suitable for automatically determining the type of capsules, tablets, and the like that are packaged.

  In pharmacies and hospitals, in general, medicine prescribed to a patient is packaged for each dose and provided to the patient. At this time, the medicine is packaged by a pharmacist selecting a medicine according to a prescription, and filling the bag by a manual operation or a packaging machine. By the way, human work inevitably causes a human error as a risk. However, in the medicine packaging work, since the patient takes the medicine, the human error is not allowed. Therefore, after the packaging, it is necessary to inspect the type of medicine packaged.

  As an inspection apparatus for inspecting the type of packaged medicine, one disclosed in Patent Document 1 is known. FIG. 6 shows a drug dispensing device for packaged medicine described in Patent Document 1 (see Patent Document 1 and Example 2). The drug inspection device is a device that performs drug inspection of a medicine (tablet or capsule) packaged in a transparent bag 101.

  The imaging unit 100 of this drug testing device includes a digital camera 102 installed above, a digital camera 103 installed below, and a parallel light source installed above, with a bag 101 containing a packaged medicine in between. And an illuminator 104 that irradiates the light downward.

  The upper digital camera 102 outputs an image obtained by color photographing the medicine packaged in the transparent bag 101. In addition, the digital camera 103 installed below outputs a silhouette image in which each medicine is captured as a gray or black silhouette image under illumination of parallel rays by the illuminator 104.

  When collating medicines, it is necessary to cut out each medicine image from the captured image. In this drug testing device, a medicine image is cut out using a silhouette image. First, binarization processing is performed on the silhouette image. Next, 8-neighbor contraction processing is performed on the binarized silhouette image. Thereby, since the medicine which contacted isolate | separates, each medicine image can be isolate | separated and cut out.

  Each medicine is collated using a color image taken by the upper digital camera 102. First, each medicine image is cut out from the color image photographed by the digital camera 102 according to the area cut out by the silhouette image first. Then, an RGB density value is detected for each drug image, and a color template is created. Further, the color image of each medicine is binarized, and the length of the long axis L and the short axis S is calculated by the number of pixels. Further, for each medicine, the area Σ of the image is calculated by the total number of pixels.

  Finally, using these obtained parameters, a collation determination value v is calculated by the following equation (1).

If the reference data and the test drug match, v becomes a small value, and if they do not match, v becomes a large value. Therefore, the drug can be verified by determining this v as a threshold value.
JP 2008-18230 A

  However, there are a great variety of drugs handled at actual dispensing sites, and many of these drugs are similar in shape, color, and pattern. Therefore, as in the above-described conventional packaged drug inspection device, when the drug is determined based on only the RGB value, the long axis, the short axis, and the plane area, the drug can be determined with sufficient accuracy. It is not possible to cause a false determination. In particular, since it is required to completely prevent accidents in which a drug is mistakenly administered to a patient at a pharmacy site, higher discrimination accuracy is required.

  On the other hand, in a dispensing site, it is necessary to prescribe a drug to a large number of patients in a short time, and it is necessary to quickly package and inspect the drug. Therefore, it is necessary that the drug type discrimination test can be easily performed.

  Accordingly, an object of the present invention is to provide a granular material type inspection capable of realizing a high degree of accuracy in determining and inspecting the type of granular article and easily performing a product type identification inspection. To provide an apparatus.

(I) Rules for Symbols First, symbols used in this specification are defined as follows.
N: the number of articles to be inspected placed on the mounting plate (N ≧ 1);
G ⁽ⁿ⁾ : Images taken by the nth imaging device (camera) (n = 1, 2, 3);
G _i ⁽ⁿ⁾ : a cut-out image of the i-th (i = 1,..., N) article cut out from the image G ⁽ⁿ⁾ ;
G _i : any one of the cropped images G _i ⁽¹⁾ , G _i ⁽²⁾ and G _i ⁽³⁾ (i = 1,..., N);
S _i : Stereo image generated from the cut-out images G _i ⁽¹⁾ and G _i ⁽²⁾ T ₁ (p): Template two-dimensional image of the article p;
T ₂ (p): template stereo image of article p;
T ₃ (p): template print image of article p;
T ₄ (p): template stamp image of article p;
Q _i : a pattern image (printed image or stamped image) extracted from the cut image G _i (i = 1,..., N);
A _i = {p _ik ^(A) }: a first candidate variety set for the cut image G _i (i = 1,..., N);
B _i = {p _ik ^(B) }: second candidate variety set for the cut image G _i (i = 1,..., N);
C _i = {p _ik ^(C) }: a third candidate variety set for the cut image G _i (i = 1,..., N);
E _i = {p _ik ^(E) }: a fourth candidate variety set for the cut image G _i (i = 1,..., N);
P _i = {p _ik }: a set of common candidate varieties for the cut image G _i (i = 1,..., N);
(P _1α1 ,..., P _NαN ): permutation of common candidates (permutation);
L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p)): a two-dimensional image distance between the cut-out image G _i (i = 1,..., N) and the template two-dimensional image T ₁ (p);
L ₂ (S _i , T ₂ (p)): stereo image distance between the stereo image S _i (i = 1,..., N) and the template two-dimensional image T ₁ (p);
L ₃ (Q _i , T ₃ (p _iαi )): Print image distance from print image;
L ₄ (Q _i , T ₄ (p _iαi )): engraved image distance from the engraved image;
D ₁ : first distance sum (= Σ _i L ₁ (G _i , T ₁ (p));
D ₂ : second distance sum (= Σ _i L ₂ (S _i , T ₂ (p));
D ₃ : third distance sum (= Σ _i L ₃ (Q _i , T ₃ (p _iαi )));
D ₄ : fourth distance sum (= Σ _i L ₄ (Q _i , T ₄ (p _iαi )));
c ₁ , c ₂ , c ₃ , c ₄ : weighting factors of the weighted sum.

(II) Configuration of the Present Invention The first configuration of the granular material type inspection apparatus of the present invention is a mounting plate for mounting an article to be inspected,
A first imaging device for imaging the article to be inspected placed on the placing plate from above obliquely;
A second imaging device for imaging an article to be inspected placed on the mounting plate from an obliquely upward direction in a direction different from that of the first imaging device;
A weigher for detecting the total weight M of the article to be inspected placed on the placing plate;
N images (N ≧ 1) of articles to be inspected placed on the mounting plate are cut out from the images picked up by the first and second image pickup devices, and cut out images {G ₁ ^(N) ,..., G _N ⁽ⁿ⁾ } (n = 1, 2: n is the number of the imaging device);
A template two-dimensional image {T ₁ (p)} obtained by photographing a plurality of types of articles to be inspected for each type of product, and a product of the product type p. Template storage means for storing the stereo image {T ₂ (p)} and the weight {m (p)} per article of the product type in association with the product type p;
Each template two-dimensional image T ₁ (p) stored in the template storage means and each cut-out image G _i ⁽ⁿ⁾ (i = 1,..., N ⁾ captured by the first or second imaging device ) And a distance between the template two-dimensional image and the cut-out image (hereinafter referred to as “two-dimensional image distance”) L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p)) is predetermined. A two-dimensional image matching means for extracting a set of candidate product types that are equal to or less than a threshold value (hereinafter, “first candidate product type set”) A _i (i = 1,..., N);
N pairs of the cutout images {(G ₁ ⁽¹⁾ , G ₁ ⁽²⁾ ),..., (G _N ⁽¹⁾ , G _N ^{(2 )} captured by the first and second imaging devices] ⁾ )}, A stereo image synthesizing means for synthesizing stereo images {S ₁ ,..., S _N } of _N inspected articles;
Stereo matching between each template stereo image T ₂ (p) stored in the template storage means and each stereo image S _i (i = 1,..., N) synthesized by the stereo image synthesis means. And a set of candidate product types whose distance between the template stereo image and the stereo image (hereinafter “stereo image distance”) L ₂ (S _i , T ₂ (p)) is equal to or less than a predetermined threshold (Hereinafter “second candidate product set”) stereo matching means for extracting B _i (i = 1,..., N);
For each of the N cutout images G _i (i = 1,..., N), the first candidate variety set A _i and the second candidate variety set B _i are commonly used. A common candidate extraction means for extracting a set of contained product types (hereinafter, “common candidate product type set”) P _i = A _i ∩B _i = {p _ik };
From the permutations (p _1α1 ,..., P _NαN ) of common candidates obtained by collecting one product type from each of the _N common candidate product types sets P _{1 ,.} , The total weight M _{j obtained by} adding the weights per article of the product types stored for the common candidate permutation (p _1α1 ,..., P _NαN ), and the total weight M detected by the weigher Weight determination means for extracting a permutation of common candidates for which the difference | M _j −M | is equal to or less than a predetermined threshold value ε _M ;
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template two-dimensional image T ₁ (p _iαi ) corresponding to each product type p _iαi and a corresponding cut-out. The first distance sum D ₁ = Σ _i L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p _iαi )) is calculated by adding the distance to the image G _i ⁽ⁿ⁾ with respect to the permutations of the common candidates. A distance sum calculation means,
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template stereo image T ₂ (p _iαi ) corresponding to each product type p _iαi and a stereo image corresponding thereto. A second distance sum calculation means for calculating a second distance sum D ₂ = Σ _i L ₂ (S _i , T ₂ (p _iαi )) obtained by adding the distance to S _i for the permutation of the common candidates;
Of the permutations {(p _1α1 ,..., P _NαN )} of the common candidates extracted by the weight determination means, the weighted sum c _{1 of} the first distance sum D ₁ and the second distance sum D ₂ D ₁ + c ₂ Comprehensive determination means for extracting a permutation of common candidates that minimizes D ₂ , and outputting as a set of inspected articles placed on the mounting plate,
It is provided with.

  According to this configuration, in addition to the two-dimensional image matching, by determining the drug by stereo matching, the height of the article is also matched, and the determination accuracy can be improved. Further, since the candidates are narrowed down by the sum of the weights of the inspected articles, the discrimination error can be further reduced as compared with the case of only discrimination based on the image. Further, in the inspection, since the article to be inspected is simply placed on the mounting table and photographed, the inspection can be performed very simply.

In the second configuration of the granular material type inspection apparatus of the present invention, in the first configuration, when the template storage means has a print on the surface of an article of the product type for each product type, When an image obtained by photographing the print (hereinafter referred to as “print image”) is stored as a template print image {T ₃ (p)} in association with the product type p, and there is a stamp on the surface of the product of the product type , An image (hereinafter referred to as “engraved image”) obtained by photographing the engraving is stored as a template engraved image {T ₄ (p)} in association with the product type p,
For each of the N cutout images G _i ⁽ⁿ⁾ (i = 1,..., N), the printed image or the stamped image (hereinafter referred to as “printed image” ^{) on} the surface of the article in the cut out image G _i ⁽ⁿ⁾ . Pattern image extracting means for determining whether or not the “pattern image” exists, and extracting the pattern image Q _i if the pattern image exists;
Printing / stamping discriminating means for discriminating whether the pattern image Q _i extracted by the pattern image extracting means is a printed image or a stamped image;
When the pattern image Q _i is discriminated as the print image by the print / engraving discrimination means, image matching between the template print images T ₃ (p) stored in the template storage means and the print images Q _i is performed. And a set of candidate product types whose print image distance L ₄ (Q _i , T ₃ (p)), which is the distance between the template print image and the print image, is equal to or less than a predetermined threshold is Print image matching means for extracting as candidate variety set C _i (i = 1,..., N);
When the pattern image Q _i is discriminated as the imprinted image by the printing / engraving discriminating means, image matching between the template imprinted image T ₄ (p) stored in the template storage means and the imprinted image Q _i is performed. And a set of candidate product types for which a stamped image distance L ₄ (Q _i , T ₄ (p)), which is a distance between the template stamped image and the stamped image, is equal to or smaller than a predetermined threshold is set to the fourth Engraving image matching means for extracting as a candidate set E _i (i = 1,..., N) of
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template print image T ₃ (p _iαi ) corresponding to each product type p _iαi and a print image Q corresponding thereto. a third distance sum calculating means for calculating a third distance sum D ₃ = Σ _i L ₃ (Q _i , T ₃ (p _iαi )) obtained by adding the distance to _i for the common candidate permutation;
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template stamp image T ₄ (p _iαi ) corresponding to each product type p _iαi and a stamp image Q corresponding thereto. a fourth distance sum calculation means for calculating a fourth distance sum D ₄ = Σ _i L ₄ (Q _i , T ₄ (p _iαi )) obtained by adding the distance to _i with respect to the permutations of the common candidates;
With
The comprehensive determination unit includes the first distance sum D ₁ and the second distance sum D ₂ among the permutations {(p _1α1 ,..., P _NαN )} of the common candidates extracted by the weight determination unit. , The third distance sum D ₃ , and the weighted sum of the fourth distance sum D ₄ c ₁ D ₁ + c ₂ D ₂ + c ₃ D ₃ + c ₄ D ₄ (c ₁ , c ₂ , c ₃ , c ₄ Is a common candidate permutation having the smallest weight constant), and outputs it as a set of articles to be inspected placed on the mounting plate.

  According to this configuration, the discrimination accuracy can be further increased by using the image matching of printing and engraving together.

In the third configuration of the granular material type inspection device of the present invention, in the second configuration, the mounting plate is made of a translucent plate material,
A third imaging device that images the article to be inspected placed on the placement plate from below or obliquely below the placement plate through the placement plate;
The region separating means extracts the cut image {G ₁ ⁽ⁿ⁾ ,..., G _N ⁽ⁿ⁾ } (n = ⁿ⁾ from the images picked up by the first, second, and third image pickup devices. 1, 2, 3: n is the number of the imaging device)
The two-dimensional image matching means is configured to extract each of the template two-dimensional images T _1k (k = 1, 2,...) And the first, second, or third imaging device. The two-dimensional image matching with the image G _i ⁽ⁿ⁾ (i = 1,..., N) is executed, and the _first two-dimensional image distance L ₁ (G _i ⁽ⁿ⁾ , T _1k ) is increased in ascending order. A candidate product set A _i (i = 1,..., N) is extracted.

  According to this configuration, an image obtained by photographing the medicine from the lower surface by the third imaging device is used in combination, so that even when the printing surface or the marking surface is facing downward, the image matching of the printing or the marking is performed correctly. And the discrimination accuracy can be further improved.

According to a fourth configuration of the granular material type inspecting apparatus of the present invention, in the second or third configuration, the common candidate extraction unit includes N pieces of the extracted images G _i (i = 1,..., N). For each of
When the cut image G _i includes a print image as the pattern image Q _i , the first candidate variety set A _i , the second candidate variety set B _i, and the third candidate variety A set of product varieties included in common with the set C _i is extracted as the common candidate product set P _i = A _i ∩B _i ∩C _i ,
When the cut image G _i includes a stamped image as the pattern image Q _i , the first candidate variety set A _i , the second candidate variety set B _i, and the fourth candidate variety A set of product varieties included in common with the set E _i is extracted as the common candidate product set P _i = A _i ∩B _i ∩E _i ,
When the cut-out image G _i does not include the pattern image Q _i , the product varieties included in both the first candidate variety set A _i and the second candidate variety set B _i The set is extracted as the common candidate variety set P _i = A _i ∩B _i .

  As described above, according to the present invention, the article to be inspected is photographed from two directions, and in addition to the two-dimensional image matching, the medicine is discriminated by stereo matching and discriminated by the sum of the weights of the inspected articles. By narrowing down candidates, it is possible to provide a granular material type inspecting apparatus having high discrimination accuracy when discriminating and inspecting the type of granular article. In addition, since the inspection work is only performed by placing the article to be inspected on the mounting plate and photographing it, it is possible to easily perform the discrimination inspection of the product type.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a granular material type inspection apparatus 1 according to Embodiment 1 of the present invention. The granular material type inspection apparatus 1 of this embodiment discriminates and inspects the type of an article B to be inspected that is packaged in a transparent packaging bag A. In this case, the inspected article B is a granular medicine such as a tablet or a capsule.

  The granular material type inspection device 1 is composed of two parts, a photographing unit 2 and a control device 3. The photographing unit 2 is a base portion of a device that photographs and weighs the article B to be inspected. The control device 3 is a base portion of a device that performs the discrimination inspection of the inspected article B based on the image information and the weighing information of the inspected article B output from the imaging unit 2. As the control device 3, a general-purpose computer in which a control program is installed can be used.

  The photographing unit 2 includes a mounting plate 4, a weigher 5, a first illumination 6, a second illumination 7, a third illumination 8, a first camera 9, a second camera 10, and a third camera 11.

  The mounting plate 4 is a horizontal table for mounting the inspected article B packaged in the transparent packaging bag A, and is made of a translucent plate material. Further, a black light-shielding plate 4 a having a square opening formed in a portion facing the portion on which the article B to be inspected is placed is placed on the lower surface of the placement plate 4. The light shielding plate 4a is provided for designating the position where the article B to be inspected packaged in the packaging bag A is to be placed, and for facilitating removal of an excessive portion of the image to be photographed. .

  Note that one or more articles B to be inspected are packaged in the packaging bag A, and the number of articles B to be inspected packaged in the packaging bag A is denoted as N (≧ 1).

  The weigher 5 detects the total weight M of the inspected article B placed on the placement plate 4. An electronic balance or the like is used as the weigher 5.

  The first illumination 6 and the second illumination 7 are illuminators installed above the placement plate 4, and obliquely inspect the article B to be inspected packaged in the packaging bag A placed on the placement plate 4. Illuminated from above. The 1st illumination 6 and the 2nd illumination 7 are installed in the position which becomes right and left with respect to the center point of the part (opening part of the light-shielding plate 4a) which mounts the to-be-inspected goods B of the mounting plate 4. FIG. The third illumination 8 is an illuminator installed below the placement plate 4 and illuminates the inspection object B packaged in the packaging bag A placed on the placement plate 4 from obliquely below. To do.

  The first camera 9 and the second camera 10 are imaging devices installed above the placement plate 4, and obliquely inspect the inspection object B packaged in the packaging bag A placed on the placement plate 4. Take an image from above. The first camera 9 and the second camera 10 are installed at positions on the left and right of the center point of the portion (the opening of the light shielding plate 4a) where the article B to be inspected is placed. Therefore, the second camera 10 captures an image of the inspected article B from obliquely above in a direction different from that of the first camera 9. The third camera 11 is an imaging device installed below the placement plate 4, and images the inspected article B packaged in the packaging bag A placed on the placement plate 4 from obliquely below. To do. For the first camera 9, the second camera 10, and the third camera 11, an electronic imaging device such as a CCD element is used.

  FIG. 2 is a functional block diagram of the granular material type inspection apparatus 1 according to the first embodiment. In FIG. 2, the same parts as those in FIG.

  The control device 3 of the granular material type inspection apparatus 1 includes a template storage unit 20, a first candidate group storage unit 21, a second candidate group storage unit 22, a third candidate group storage unit 23, a fourth candidate group storage unit 24, Common candidate storage unit 25, common candidate permutation storage unit 26, region separation module 27, image selection module 28, two-dimensional image matching module 29, stereo image synthesis module 30, stereo matching module 31, pattern image extraction module 32, Illumination controller 33, printing / engraving discrimination module 34, printed image matching module 35, engraved image matching module 36, common candidate extraction module 37, weight determination module 38, distance sum calculation module 39, distance sum calculation module 40, distance sum Calculation module 41, distance sum calculation module 42, total size And a module 43, an output module 44, and an output device 45. Each storage unit and each module other than the output device 45 are coded by a program, and are functionally realized in the control device 3 by the control device 3 loading and executing the program. As the output device 45, an output device such as a display or a speaker is used.

The template storage unit 20 includes, for each product type, a plurality of types of articles to be inspected, a template two-dimensional image {T ₁ (p)} of the product type p, and a template stereo image {T ₂ (p)}. , The template print image {T ₃ (p)}, the template stamp image {T ₄ (p)}, and the weight {m (p)} per article of the product type are stored in association with the product type p. ing.

Here, the “template two-dimensional image” is a two-dimensional image obtained by photographing an article used as a reference pattern from directly above. A “template stereo image” is a stereo image of an article used as a reference pattern. The “template print image” is a print image obtained by photographing a print on the surface of an article used as a reference pattern. The “template stamp image” is a stamp image obtained by photographing a stamp on the surface of an article used as a reference pattern. As defined in the above rules, each template two-dimensional image is T ₁ (p), each template stereo image is T ₂ (p), each template print image is T ₃ (p), and each template stamp image is T _4. Let us write (p). Here, p represents a product type. Each template is an image obtained by photographing a representative article of the product type or an image synthesized therefrom, and is prepared in advance and stored in the template storage unit 20 in advance.

  Note that there is no template print image or template stamp image for the product type p without print or stamp.

The area separation module 27 individually outputs images of N inspected articles B placed on the placement plate 4 from the images captured by the first camera 9, the second camera 10, and the third camera 11. To generate a cut image {G ₁ ⁽ⁿ⁾ ,..., G _N ⁽ⁿ⁾ }. Here, G _i ⁽ⁿ⁾ (i = 1,..., N) is a clipped image, and n (= 1, 2, 3) represents the camera number. That is, n = 1 is a cut-out image cut out from the image picked up by the first camera 9, n = 2 is a cut-out image cut out from the image picked up by the second camera 10, and n = 3 is the third camera 11. The cut-out image cut out from the image imaged in FIG. Therefore, the region separation module 27 generates 3N cut images in total.

The image selection module 28 selects and outputs the cut image G _i (n) cut by the region separation module 27.

The two-dimensional image matching module 29 uses each template two-dimensional image T ₁ (p) stored in the template storage unit 20 and each clip imaged by the first camera 9, the second camera 10, or the third camera 11. Two-dimensional image matching with the image G _i is executed, and a distance L ₁ (G _i , T ₁ (p)) between the template two-dimensional image T ₁ (p) and the cut-out image G _i is calculated. This distance L ₁ (G _i , T ₁ (p)) is referred to as “two-dimensional image distance”. Further, the two-dimensional image matching module 29 generates a template two-dimensional image T ₁ (p _ik ^(A) ) in which the two-dimensional image distance L ₁ (G _i , T ₁ (p)) is equal to or less than a predetermined threshold ε _1. Then, a set A _i (i = 1,..., N) of the product varieties p _ik ^(A) corresponding to them is extracted. This set of candidate varieties A _i = {p _ik ^(A) } is referred to as “first candidate varieties set”. Here, p _ik ^(A) represents the k-th element (article) included in the first candidate variety set A _i .

The first candidate group storage unit 21 includes a first candidate variety set A _i = {p _ik ^(A) } extracted by the two-dimensional image matching module 29 and each element p _ik ^{(A) of the} set A _i. The two-dimensional image distance L ₁ (G _i , T ₁ (p _ik ^(A) )) between the template two-dimensional image T ₁ (p _ik ^(A) ) and the cut image G _i corresponding thereto is stored.

The stereo image synthesis module 30 includes N pairs of clipped images captured by the first camera 9 and the second camera 10 {(G ₁ ⁽¹⁾ , G ₁ ⁽²⁾ ), ..., (G _N ^{(1 )} , G _N ⁽²⁾ )} to synthesize stereo images {S ₁ ,..., S _N } of _N inspected articles.

The stereo matching module 31 performs stereo matching between each template stereo image T ₂ (p) stored in the template storage unit 20 and each stereo image S _i (i = 1,..., N). Then, the distance L ₂ (S _i , T ₂ (p)) between the template stereo image T ₂ (p) and the stereo image S _i is calculated. This distance L ₂ (S _i , T ₂ (p)) is referred to as “stereo image distance”. In addition, the stereo matching module 31 extracts a template stereo image T ₂ (p _ik ^(B) ) in which the stereo image distance L ₂ (G _i , T ₂ (p)) is equal to or less than a predetermined threshold ε _2. Then, a set B _i (i = 1,..., N) of product varieties p _ik ^(B) corresponding to them is extracted. This set of candidate varieties B _i = {p _ik ^(B) } is referred to as a “second candidate varieties set”. Here, p _ik ^(B) represents the k-th element (article) included in the second candidate variety set B _i .

The second candidate group storage unit 22 corresponds to the second candidate variety set B _i = {p _ik ^(B) } extracted by the stereo matching module 31 and each element p _ik ^(B) of the set B _i . The stereo image distance L ₂ (S _i , T ₂ (p _ik ^(B) )) between the template stereo image T ₂ (p _ik ^(B) ) and the corresponding stereo image S _i is stored.

Pattern image extraction module 32, N pieces of the cut image _{G i (i = 1, ...} , N) for each, whether the surface to the print image or engraved image of the article in the clipped images G _i exist Determine whether. A printed image or a stamped image is collectively referred to as a “pattern image”. If a pattern image exists, the pattern image extraction module 32 extracts the pattern image Q _i .

  The illumination controller 33 adjusts the light quantity of the 1st illumination 6, the 2nd illumination 7, and the 3rd illumination 8, or controls those lighting / extinction.

The print / stamping determination module 34 determines whether the pattern image Q _i extracted by the pattern image extraction module 32 is a print image or a stamp image.

When the pattern image Q _i is determined to be a print image by the print / engravement determination module 34, the print image matching module 35 and each template print image T ₃ (p) stored in the template storage unit 20 and the print image Image matching with Q _i is executed, and a distance L ₃ (Q _i , T ₃ (p)) between the template print image T ₃ (p) and the print image Q _i is calculated. This distance L ₃ (Q _i , T ₃ (p)) is referred to as “print image distance”. Further, the print image matching module 35 extracts a template print image T ₃ (p _ik ^(C) ) in which the print image distance L ₃ (G _i , T ₃ (p)) is equal to or less than a predetermined threshold ε ₃ , A set C _i (i = 1,..., N) of the product varieties p _ik ^(C) corresponding to them is extracted. This set of candidate varieties C _i = {p _ik ^(C) } is referred to as a “third candidate varietal set”. Here, p _ik ^(C) represents the k-th element (article) included in the third candidate variety set C _i .

The third candidate group storage unit 23 corresponds to the third candidate variety set C _i = {p _ik ^(C) } extracted by the print image matching module 35 and each element p _ik ^(C) of the set C _i . The print image distance L ₃ (Q _i , T ₃ (p _ik ^(C) )) between the template print image T ₃ (p _ik ^(C) ) and the corresponding print image Q _i is stored.

When the pattern image Q _i is determined to be a stamped image by the printing / stamping determination module 34, the stamped image matching module 36 and each template stamped image T ₄ (p) stored in the template storage unit 20 and the stamped image Image matching with Q _i is executed, and a distance L ₄ (Q _i , T ₄ (p)) between the template stamped image T ₄ (p) and the stamped image Q _i is calculated. This distance L ₄ (Q _i , T ₃ (p)) is referred to as “engraved image distance”. Further, the imprinted image matching module 36 extracts a template imprinted image T ₄ (p _ik ^(E) ) in which the imprinted image distance L ₄ (G _i , T ₄ (p)) is equal to or smaller than a predetermined threshold ε ₄ , A set E _i (i = 1,..., N) of product varieties p _ik ^(E) corresponding to them is extracted. This set of candidate varieties E _i = {p _ik ^(E) } is referred to as a “fourth candidate varieties set”. Here, p _ik ^(E) represents the k-th element (article) included in the fourth candidate variety set E _i .

The fourth candidate group storage unit 24 corresponds to the fourth candidate variety set E _i = {p _ik ^(E) } extracted by the stamped image matching module 36 and each element p _ik ^(E) of the set E _i . The stamp image distance L ₄ (Q _i , T ₄ (p _ik ^(E) )) between the template stamp image T ₄ (p _ik ^(E) ) and the corresponding stamp image Q _i is stored.

The common candidate extraction module 37 extracts a common candidate variety set P _i = {p _ik } for each of the N cut images G _i (i = 1,..., N) output from the region separation module 27. To do. Here, when the print image Q _i is included in the cut-out image G _i , the common candidate extraction module 37 performs the first candidate variety set A _i , the second candidate variety set B _i, and the third A set of product varieties included in common with the candidate product variety set C _i is extracted as a common candidate product set P _i = A _i ∩B _i ∩C _i . When the cut image G _i includes the stamped image Q _i , the common candidate extraction module 37 uses the first candidate product type set A _i , the second candidate product type set B _i, and the fourth candidate. A set of product types included in common with the product type set E _i is extracted as a common candidate product type set P _i = A _i ∩B _i ∩E _i . If the cutout image G _i does not include the pattern image Q _i , the common candidate extraction module 37 is common to the first candidate variety set A _i and the second candidate variety set B _i. The set of product types included is extracted as a common candidate product type set P _i = A _i ∩B _i .

The common candidate storage unit 25 stores the common candidate product type set P _i (i = 1,..., N) extracted by the common candidate extraction module 37.

The weight determination module 38 extracts a common product type from each of the _N common candidate product type sets P ₁ ,..., P _N stored in the common candidate storage unit 25, and collects and collects the common candidate permutations (p _1α1 ,..., P _NαN ), the total weight M _j is calculated by adding the weight per article of the product type stored in the template storage unit 20 to the permutation (p _1α1 ,..., P _NαN ) of the common candidates. . Then, a permutation of common candidates in which the difference | M _j −M | from the total weight M detected by the weigher 5 is equal to or less than a predetermined threshold ε _M is extracted.

  The common candidate permutation storage unit 26 stores permutations of common candidates extracted by the weight determination module 38.

The distance sum calculation module 39 generates a template two-dimensional image T ₁ (p _iαi ) corresponding to each product type p _iαi for each permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination module 38. ) And the corresponding two-dimensional image distance L ₁ (G _i , T ₁ (p _iαi )) between the cut-out image G _i and the common candidate permutation D ₁ = Σ _i L ₁ (G _i , T ₁ (p _iαi )) is calculated.

Distance sum calculation module 40, the weight determination each common candidates permutations extracted by module _{38 (p 1α1, ..., p} NαN) relative to the template stereo image _T 2 corresponding to each article type _{p iαi (p iαi} ) And a stereo image distance L ₂ (S _i , T ₂ (p _iαi )) between the corresponding stereo image S _i and the permutation of the common candidates, a second distance sum D ₂ = Σ _i L ₂ ( S _i , T ₂ (p _iαi )) is calculated.

The distance sum calculation module 41 generates a template print image T ₃ (p _iαi ) corresponding to each product type p _iαi for each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination module 38. And a print image distance L ₃ (Q _i , T ₃ (p _iαi )) between the corresponding print image Q _i and a corresponding permutation of the common candidate D ₃ = Σ _i L ₃ (Q _i , T ₃ (p _iαi )) is calculated.

The distance sum calculation module 42, for each permutation (p _1α1 ,..., P _NαN ) of the common candidates extracted by the weight determination module 38, is a template stamp image T ₄ (p _iαi ) corresponding to each product type p _iαi. And a stamped image distance L ₄ (Q _i , T ₄ (p _iαi )) between the corresponding stamped image Q _i and the corresponding permutation of the common candidate D ₄ = Σ _i L ₄ (S _i , T ₄ (p _iαi )) is calculated.

The overall determination module 43 includes, among the permutations {(p _1α1 ,..., P _NαN )} of the common candidates extracted by the weight determination module 38, the first distance sum D ₁ , the second distance sum D ₂ , weighted sum of the third distance sum _{D 3,} and a fourth distance sum _{D 4 c 1 D 1 + c} 2 D 2 + c 3 D 3 + c 4 D 4 (c 1, c 2, c 3, c 4 are weighting constants) Is extracted as a set S of articles to be inspected placed on the placing plate 4.

The output module 44 compares the set S of articles to be inspected output from the comprehensive determination module 43 with a set of correct articles S ₀ set in advance. If S = S ₀ , the output module 44 A matching signal is output. When S ≠ S ₀ , a mismatch signal is output to the output device 45.

  When the mismatch signal is output from the output module 44, the output device 45 notifies the user of a warning by an image or sound.

  The operation of the particulate matter type inspection apparatus 1 according to the first embodiment of the present invention configured as described above will be described below.

  FIG. 3 is a flowchart showing the operation of the granular material type inspection apparatus 1 according to the first embodiment.

The user first sets a set S ₀ = {p ₀₁ ,..., P _0m } of correct article combinations of articles to be packaged in the transparent bag A in the output module 44 in advance. Then, the transparent transparent bag A in which the article to be inspected B is packaged is placed on the mounting plate 4. As an example, it is assumed that the inspected article B is a granular medicine such as a tablet or a capsule as shown in FIG.

First, in step S1, the illumination controller 33 turns on the first illumination 6, the second illumination 7, and the third illumination 8, and the first camera 9, the second camera 10, and the third camera 11 are mounted plates. An image of the inspected article B packaged in the transparent bag A placed on the image 4 is taken. Images taken by the first camera 9, the second camera 10, and the third camera 11 in accordance with the code convention are denoted as G ⁽¹⁾ , G ⁽²⁾ , G ⁽³⁾ . Each image is output to the region separation module 27.

  At this time, in order to clearly highlight the image of the inscription on the surface of the article B to be inspected, the illumination controller 33 turns on only the first illumination 6 or only the second illumination 7 and turns on the first camera 9 and It is also possible to perform shooting with the second camera 10.

Next, in step S2, the region separation module 27 cuts out a partial image of the inspected article B included in each of the images G ⁽¹⁾ , G ⁽²⁾ , and G ⁽³⁾ . The partial images obtained in this way are cut out images {G ₁ ⁽ⁿ⁾ ,..., G _N ⁽ⁿ⁾ } (n = 1, 2, 3). For example, when the granular medicine shown in FIG. 4 is packaged in the transparent bag A as the article B to be inspected, N = 7, and thus 7 × 3 = 21 cutout images are generated in total. Each cut-out image is labeled, and the same article label (number) is attached to the cut-out image of the same article taken by different cameras. When the cut image is ^denoted as G _i ⁽ⁿ⁾ , i represents an article label.

Next, in step S3, the image selection module 28 selects G _i ⁽¹⁾ , G _i ⁽²⁾ , G _i ⁽³⁾ as the cut-out images in ascending order of the article label i.

Next, in step S < ^b > 4, the two-dimensional image matching module 29 performs the cut-out images G _i ⁽¹⁾ , G _i ⁽²⁾ , G _i ⁽³⁾ and the template two-dimensional images stored in the template storage unit 20. Two-dimensional image matching with T ₁ (p) is executed, and a two-dimensional image distance L ₁ (G _i ⁽ⁿ⁾ , T ₁ between the template two-dimensional image T ₁ (p) and the cut-out image G _i ^(n). (P)) is calculated. Here, the two-dimensional image matching need not be performed for all of the cut-out images G _i ⁽¹⁾ , G _i ⁽²⁾ , and G _i ^(3). Good. Therefore, for example, when template two-dimensional images T ₁ (p) of M articles p are stored in the template storage unit 20, M times of two-dimensional image matching are executed. For the two-dimensional image matching, a known method can be used. For example, an elastic matching algorithm such as DP matching is used.

The two-dimensional image distance L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p)) is expressed by the following equation.

Here, Φ represents an affine transformation. ‖ / ‖ Represents a distance (norm), and ‖T ₁ (p) −Φ (G _i ⁽ⁿ⁾ ) ‖ is a distance between corresponding points of the image T ₁ (p) and the image Φ (G _i ⁽ⁿ⁾ ). Is the average value.

Then, the two-dimensional image matching module 29 outputs a template two-dimensional image T ₁ (p _ik ^(A ) in which the two-dimensional image distance L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p)) is equal to or less than a predetermined threshold ε _1. ⁾ ) _Is extracted, and a set of product varieties p _ik ^(A) corresponding to them (first candidate product type set) A _i is extracted. That is, the first candidate variety set A _i is expressed as follows.

The extracted first candidate variety set A _i and the two-dimensional image distance L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p)) for each element p _ik ^(A ) are the first candidate group storage unit 21 is stored.

Next, in step S < ^b > 5, the stereo image synthesis module 30 determines from the N sets of clipped image pairs (G _i ⁽¹⁾ , G _i ⁽²⁾ ) captured by the first camera 9 and the second camera 10. A stereo image S _{i of the} article to be inspected is synthesized. Next, the stereo matching module 31 performs stereo matching between each template stereo image T ₂ (p) stored in the template storage unit 20 and each stereo image S _i, and the template stereo image T _2. A stereo image distance L ₂ (S _i , T ₂ (p)) between (p) and the stereo image S _i is calculated. Also for stereo matching, a known method can be used, and for example, an elastic matching algorithm such as DP matching is used.

The stereo image distance L ₂ (S _i , T ₂ (p)) is expressed by the following equation.

Then, the stereo matching module 31 extracts a template stereo image T ₂ (p _ik ^(B) ) in which the stereo image distance L ₂ (G _i , T ₂ (p)) is equal to or less than a predetermined threshold ε _2. Then, a set of product varieties p _ik ^(B) corresponding to them (second candidate product type set) B _i is extracted. That is, the second candidate variety set B _i is expressed as follows.

The extracted second candidate variety set B _i and the stereo image distance L ₂ (S _i , T ₂ (p _ik ^(B) )) for each element p _ik ^(B ) are the second candidate group storage unit 22 is stored.

Next, in step S ^{< b >} 6, the pattern image extraction module 32 performs the surface of the article in each cut-out image G _i ^{(n) for} the cut-out images G _i ⁽¹⁾ , G _i ⁽²⁾ , G _i ^(3). It is determined whether or not there is a pattern image. If a pattern image exists, the pattern image extraction module 32 extracts the pattern image Q _i .

FIG. 5 shows an example of an article image including a pattern image. FIG. 5A shows a case where the surface of the article is printed, and FIGS. 5B and 5C show a case where the surface of the article is marked. As shown in FIG. 5C, in many cases, printing or engraving is applied only to one surface of an article. Therefore, by extracting the pattern image Q _i using the cut-out image G _i ⁽¹⁾ or G _i ⁽²⁾ on the front side surface and the cut-out image G _i ⁽³⁾ on the back side, the pattern image Q can be surely obtained. _i can be extracted.

When the pattern image is not extracted, the pattern image extraction module 32 causes the illumination controller 33 to turn on only one of the first illumination 6 or the second illumination 7 and the first camera 9 and the second camera. 10 may be performed again, the cut-out images G _i ⁽¹⁾ and G _i ⁽²⁾ may be re-created, and the pattern image may be extracted again. In this way, even when the marking on the surface of the article to be inspected B is unclear, the image of the marking on the surface of the article to be inspected B can be clearly raised. It becomes possible to extract reliably.

Next, when the pattern image Q _i is extracted by the pattern image extraction module 32 in step S7, the process proceeds to the next step S8, and when the pattern image is not extracted, the process proceeds to step S11.

Next, in step S8, the printing / stamping determination module 34 determines whether the pattern image Q _i extracted by the pattern image extraction module 32 is a printing image or a stamping image. Here, whether the image is a printed image or a stamped image can be determined by determining the gradient of the edge of the pattern image. In other words, the gradient of the edge of the pattern image is steep in the case of a printed image, and the gradient of the edge of the pattern image is gentle in the case of a stamped image. Accordingly, whether the pattern image is a printed image or a stamped image is determined by determining a threshold value for the average value of the edge portions of the edge image of the pattern image. If the pattern image Q _i is a print image, the process proceeds to step S9. If the pattern image Q _i is a stamped image, the process proceeds to step S10.

If the pattern image Q _i is a print image, in step S9, the print image matching module 35 executes image matching between each template print image T ₃ (p) stored in the template storage unit 20 and the print image Q _i. Then, a print image distance L ₃ (Q _i , T ₃ (p)) between the template print image T ₃ (p) and the print image Q _i is calculated.

The print image distance L ₃ (Q _i , T ₃ (p)) is expressed by the following equation.

Then, the print image matching module 35 extracts a template print image T ₃ (p _ik ^(C) ) in which the print image distance L ₃ (Q _i , T ₃ (p)) is equal to or less than a predetermined threshold ε ₃ , A set of product varieties p _ik ^(C) corresponding to them (third candidate variety set) C _i is extracted. That is, the third candidate variety set C _i is expressed as follows.

The extracted third candidate variety set C _i and the print image distance L ₃ (Q _i , T ₃ (p)) for each element p _ik ^(C) are stored in the third candidate group storage unit 23. Is done.

On the other hand, when the pattern image Q _i is a stamp image, in step S10, the stamp image matching module 36 performs image matching between each template stamp image T ₄ (p) stored in the template storage unit 20 and the stamp image Q _i. Is executed, and a stamped image distance L ₄ (Q _i , T ₄ (p)) between the template stamped image T ₄ (p) and the stamped image Q _i is calculated.

The stamped image distance L ₄ (Q _i , T ₄ (p)) is expressed by the following equation.

Then, the stamped image matching module 36 extracts a template stamped image T ₄ (p _ik ^(E) ) in which the stamped image distance L ₄ (G _i , T ₄ (p)) is equal to or less than a predetermined threshold ε ₄ . A set (fourth candidate variety set) E _i (i = 1,..., N) of the product varieties p _ik ^(E) corresponding to them is extracted. That is, the fourth candidate variety set E _i is expressed as follows.

The extracted fourth candidate variety set E _i and the stamped image distance L ₄ (Q _i , T ₄ (p)) for each element p _ik ^(E) are stored in the fourth candidate group storage unit 24. Is done.

Next, in step S11, the common candidate extraction module 37 extracts a common candidate variety set P _i = {p _ik } for the cut image G _i . This extraction process is performed as follows depending on the presence or absence of a printed image or a stamped image.

(1) When the cut image G _i includes the print image Q _{i The} common candidate extraction module 37 performs the first candidate variety set A _i , the second candidate variety set B _i, and the third candidate variety. A set of product types included in common with the set C _i is extracted as a common candidate product type set P _i .

(2) When the cut image G _i includes the stamped image Q _{i The} common candidate extraction module 37 performs the first candidate variety set A _i , the second candidate variety set B _i, and the fourth candidate variety. A set of product types included in common with the set E _i is extracted as a common candidate product type set P _i .

(3) When the cutout image G _i does not include the pattern image Q _{i The} common candidate extraction module 37 is included in both the first candidate variety set A _i and the second candidate variety set B _i extracting a set of the ones varieties common candidate varieties set P _i.
That is, the common candidate variety set P _i is as follows.

The extracted common candidate variety set _Pi is stored in the common candidate storage unit 25.

Next, in step S12, for all the clipped images G _i, and determines whether the processing of steps S3~S11 is completed, if not completed the process returns to step S3, if completed, The process proceeds to the next step S13.

  Next, in step S <b> 13, the weight determination module 38 acquires the total weight M of the inspected article B output from the weigher 5.

Next, in step S14, the weight determination module 38 takes out and collects one product type from each of the _N common candidate product type sets P ₁ ,..., P _N stored in the common candidate storage unit 25. A common candidate permutation o (α ₁ ,..., Α _N ) = (p _1α1 ,..., P _NαN ) is _created . Then, for each element p _iαi of this permutation o (α ₁ ,..., Α _N ), the weight w (p _iαi ) per article of the product type stored in the template storage unit 20 is acquired, and their weights The total weight M _j (α ₁ ,..., α _N ) obtained by adding w (p _iαi ) is calculated.

Then, the weight determination module 38 generates a common candidate permutation in which the difference | M _j (α ₁ ,..., Α _N ) −M | from the total weight M detected by the weigher 5 is equal to or less than a predetermined threshold value ε _M. A set Os is extracted. That is, the permutation set Os is as follows.

Next, in step S15, for each element o (α ₁ ,..., Α _N ) of the permutation set Os, the distance sum calculation modules 39, 40, 41, and 42 respectively receive the first distance sum D ₁ (α ₁ ,..., Α _N ), second distance sum D ₂ (α ₁ ,..., Α _N ), third distance sum D ₃ (α ₁ ,..., Α _N ), fourth distance sum D ₄ (α ₁ ,..., Α _N ) is calculated by the following equation.

Next, the comprehensive determination module 43 uses a first distance sum D ₁ (α ₁ ,..., Α _N ) and a second distance sum D ₂ for each element o (α ₁ ,..., Α _N ) of the permutation set Os. (Α ₁ ,..., Α _N ), a third distance sum D ₃ (α ₁ ,..., Α _N ), and a fourth distance sum D ₄ (α ₁ ,..., Α _N ), a weighted sum D (α ₁ ,..., Α _N ) is calculated by the following equation.

Here, c ₁ , c ₂ , c ₃ , and c ₄ are weight constants, and may be determined to be optimal values experimentally.

Then, the comprehensive determination module 43 extracts a permutation o (β ₁ ,..., Β _N ) of common candidates that minimizes the weighted sum D (α ₁ ,..., Α _N ), and is placed on the placement plate 4. Output as a set S of inspected articles. That is, the set S is as follows.

Finally, in step S16, the output module 44 compares the set S of the article to be inspected for total determination module 43 outputs, and a set S ₀ of the correct article has been set in advance, in the case of S = S ₀ Outputs a matching signal to the output device 45, and outputs a mismatch signal to the output device 45 when S ≠ S ₀ . When the mismatch signal is output from the output module 44, the output device 45 notifies the user of a warning by an image or sound.

  As described above, in the granular material type inspection apparatus 1 according to the present embodiment, in addition to two-dimensional image matching, the drug is determined by stereo matching, and the determination candidates are narrowed down by the sum of the weights of the articles to be inspected. Therefore, it is possible to improve the discrimination accuracy when discriminating and inspecting the type of granular article. Further, when the article is printed or engraved, the accuracy can be further improved by performing collation of the printing or engraving together.

In this embodiment, the print image matching module 35 and the stamped image matching module 36 are provided to perform matching of the print image and the stamped image. However, depending on the application, printing or stamping may be performed on the inspected article. Sometimes it is clear from the beginning that is not attached. In such a case, the pattern image extraction module 32, the printing / engraving discrimination module 34, the printing image matching module 35, and the engraving image matching module 36 can be omitted. When these modules are omitted, the common candidate extraction module 37 is always common to the first candidate variety set A _i and the second candidate variety set B _i as the common candidate variety set P _i. Change to extract a set of included product types. That is, the common candidate variety set _Pi is as follows.

  In the present embodiment, the inspection of the granular medicine is described as an example, but the granular material type inspection apparatus of the present invention can also be applied to granular inspected articles other than the granular medicine.

It is a figure which shows the structure of the granular material kind inspection apparatus which concerns on Example 1 of this invention. It is a functional block diagram of the granular material kind inspection apparatus 1 which concerns on Example 1. FIG. 3 is a flowchart showing the operation of the granular material type inspection apparatus 1 according to the first embodiment. It is an example of inspected article B. It is an example of the article | item image containing a pattern image. It is a figure which shows the medicine inspection apparatus of the packaging medicine of patent document 1.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Granules kind inspection apparatus 2 Image pick-up unit 3 Control apparatus 4 Mounting plate 4a Light-shielding plate 5 Weighing instrument 6 First illumination 7 Second illumination 8 Third illumination 9 First camera 10 Second camera 11 Third camera 20 Template storage Unit 21 first candidate group storage unit 22 second candidate group storage unit 23 third candidate group storage unit 24 fourth candidate group storage unit 25 common candidate storage unit 26 common candidate permutation storage unit 27 region separation module 28 image selection module 29 Dimensional image matching module 30 Stereo image synthesis module 31 Stereo matching module 32 Pattern image extraction module 33 Illumination controller 34 Printing / engraving discrimination module 35 Printed image matching module 36 Engraving image matching module 37 Common candidate extraction module 38 Weight determination Module 39, 40, 41, 42 Distance Calculation module 43 Overall judgment module 44 the output module 45 output apparatus A packaging bag B the article to be inspected

Claims (4)

A mounting plate for mounting the article to be inspected;
A first imaging device for imaging the article to be inspected placed on the placing plate from above obliquely;
A second imaging device for imaging an article to be inspected placed on the mounting plate from an obliquely upward direction in a direction different from that of the first imaging device;
A weigher for detecting the total weight M of the article to be inspected placed on the placing plate;
N images (N ≧ 1) of articles to be inspected placed on the mounting plate are cut out from the images picked up by the first and second image pickup devices, and cut out images {G ₁ ^(N) ,..., G _N ⁽ⁿ⁾ } (n = 1, 2: n is the number of the imaging device);
A template two-dimensional image {T ₁ (p)} obtained by photographing a plurality of types of articles to be inspected for each type of product, and a product of the product type p. Template storage means for storing the stereo image {T ₂ (p)} and the weight {m (p)} per article of the product type in association with the product type p;
Each template two-dimensional image T ₁ (p) stored in the template storage means and each cut-out image G _i ⁽ⁿ⁾ (i = 1,..., N ⁾ captured by the first or second imaging device ) And a distance between the template two-dimensional image and the cut-out image (hereinafter referred to as “two-dimensional image distance”) L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p)) is predetermined. A two-dimensional image matching means for extracting a set of candidate product types that are equal to or less than a threshold value (hereinafter, “first candidate product type set”) A _i (i = 1,..., N);
N pairs of the cutout images {(G ₁ ⁽¹⁾ , G ₁ ⁽²⁾ ),..., (G _N ⁽¹⁾ , G _N ^{(2 )} captured by the first and second imaging devices] ⁾ )}, A stereo image synthesizing means for synthesizing stereo images {S ₁ ,..., S _N } of _N inspected articles;
Stereo matching between each template stereo image T ₂ (p) stored in the template storage means and each stereo image S _i (i = 1,..., N) synthesized by the stereo image synthesis means. And a set of candidate product types whose distance between the template stereo image and the stereo image (hereinafter “stereo image distance”) L ₂ (S _i , T ₂ (p)) is equal to or less than a predetermined threshold (Hereinafter “second candidate product set”) stereo matching means for extracting B _i (i = 1,..., N);
For each of the N cutout images G _i (i = 1,..., N), the first candidate variety set A _i and the second candidate variety set B _i are commonly used. A common candidate extraction means for extracting a set of contained product types (hereinafter, “common candidate product type set”) P _i = A _i ∩B _i = {p _ik };
From the permutations (p _1α1 ,..., P _NαN ) of common candidates obtained by collecting one product type from each of the _N common candidate product types sets P _{1 ,.} , The total weight M _{j obtained by} adding the weights per article of the product types stored for the common candidate permutation (p _1α1 ,..., P _NαN ), and the total weight M detected by the weigher Weight determination means for extracting a permutation of common candidates for which the difference | M _j −M | is equal to or less than a predetermined threshold value ε _M ;
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template two-dimensional image T ₁ (p _iαi ) corresponding to each product type p _iαi and a corresponding cut-out. The first distance sum D ₁ = Σ _i L ₁ (G _i ⁽ⁿ⁾ , T ₁ (p _iαi )) is calculated by adding the distance to the image G _i ⁽ⁿ⁾ with respect to the permutations of the common candidates. A distance sum calculation means,
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template stereo image T ₂ (p _iαi ) corresponding to each product type p _iαi and a stereo image corresponding thereto. A second distance sum calculation means for calculating a second distance sum D ₂ = Σ _i L ₂ (S _i , T ₂ (p _iαi )) obtained by adding the distance to S _i for the permutation of the common candidates;
Of the permutations {(p _1α1 ,..., P _NαN )} of the common candidates extracted by the weight determination means, the weighted sum c _{1 of} the first distance sum D ₁ and the second distance sum D ₂ D ₁ + c ₂ Comprehensive determination means for extracting a permutation of common candidates that minimizes D ₂ , and outputting as a set of inspected articles placed on the mounting plate,
A granular material type inspection apparatus characterized by comprising: In the template storage means, if there is a print on the surface of an article of each product type for each product type, an image obtained by photographing the print (hereinafter referred to as “print image”) is displayed as a template print image {T ₃ (p )} Is stored in association with the product type p, and if there is an inscription on the surface of the product of the product type, an image (hereinafter referred to as “engraved image”) obtained by photographing the inscription is used as a template imprinted image {T ₄ ( p)} is stored in association with the product type p,
For each of the N cutout images G _i ⁽ⁿ⁾ (i = 1,..., N), the printed image or the stamped image (hereinafter referred to as “printed image” ^{) on} the surface of the article in the cut out image G _i ⁽ⁿ⁾ . Pattern image extracting means for determining whether or not the “pattern image” exists, and extracting the pattern image Q _i if the pattern image exists;
Printing / stamping discriminating means for discriminating whether the pattern image Q _i extracted by the pattern image extracting means is a printed image or a stamped image;
When the pattern image Q _i is discriminated as the print image by the print / engraving discrimination means, image matching between the template print images T ₃ (p) stored in the template storage means and the print images Q _i is performed. And a set of candidate product types whose print image distance L ₄ (Q _i , T ₃ (p)), which is the distance between the template print image and the print image, is equal to or less than a predetermined threshold is Print image matching means for extracting as candidate variety set C _i (i = 1,..., N);
When the pattern image Q _i is discriminated as the imprinted image by the printing / engraving discriminating means, image matching between the template imprinted image T ₄ (p) stored in the template storage means and the imprinted image Q _i is performed. And a set of candidate product types for which a stamped image distance L ₄ (Q _i , T ₄ (p)), which is a distance between the template stamped image and the stamped image, is equal to or smaller than a predetermined threshold is set to the fourth Engraving image matching means for extracting as a candidate set E _i (i = 1,..., N) of
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template print image T ₃ (p _iαi ) corresponding to each product type p _iαi and a print image Q corresponding thereto. a third distance sum calculating means for calculating a third distance sum D ₃ = Σ _i L ₃ (Q _i , T ₃ (p _iαi )) obtained by adding the distance to _i for the common candidate permutation;
For each common candidate permutation (p _1α1 ,..., P _NαN ) extracted by the weight determination means, a template stamp image T ₄ (p _iαi ) corresponding to each product type p _iαi and a stamp image Q corresponding thereto. a fourth distance sum calculation means for calculating a fourth distance sum D ₄ = Σ _i L ₄ (Q _i , T ₄ (p _iαi )) obtained by adding the distance to _i with respect to the permutations of the common candidates;
With
The comprehensive determination unit includes the first distance sum D ₁ and the second distance sum D ₂ among the permutations {(p _1α1 ,..., P _NαN )} of the common candidates extracted by the weight determination unit. , The third distance sum D ₃ , and the weighted sum of the fourth distance sum D ₄ c ₁ D ₁ + c ₂ D ₂ + c ₃ D ₃ + c ₄ D ₄ (c ₁ , c ₂ , c ₃ , c ₄ 2. The granular material type inspection apparatus according to claim 1, wherein a permutation of common candidates having a minimum weight constant) is extracted and output as a set of inspected articles placed on the mounting plate. The mounting plate is made of a translucent plate material,
A third imaging device that images the article to be inspected placed on the placement plate from below or obliquely below the placement plate through the placement plate;
The region separating means extracts the cut image {G ₁ ⁽ⁿ⁾ ,..., G _N ⁽ⁿ⁾ } (n = 1, 2, 3: n is the number of the imaging device)
The two-dimensional image matching means is configured to extract each of the template two-dimensional images T _1k (k = 1, 2,...) And the first, second, or third imaging device. The two-dimensional image matching with the image G _i ⁽ⁿ⁾ (i = 1,..., N) is executed, and the _first two-dimensional image distance L ₁ (G _i ⁽ⁿ⁾ , T _1k ) is increased in ascending order. 3. The granular material type inspection apparatus according to claim 2, wherein the candidate type set A _i (i = 1,..., N) is extracted. The common candidate extraction means is configured for each of the N cut images G _i (i = 1,..., N).
When the cut image G _i includes a print image as the pattern image Q _i , the first candidate variety set A _i , the second candidate variety set B _i, and the third candidate variety A set of product varieties included in common with the set C _i is extracted as the common candidate product set P _i = A _i ∩B _i ∩C _i ,
When the cut image G _i includes a stamped image as the pattern image Q _i , the first candidate variety set A _i , the second candidate variety set B _i, and the fourth candidate variety A set of product varieties included in common with the set E _i is extracted as the common candidate product set P _i = A _i ∩B _i ∩E _i ,
When the cut-out image G _i does not include the pattern image Q _i , the product varieties included in both the first candidate variety set A _i and the second candidate variety set B _i 4. The granular material type inspection apparatus according to claim 2, wherein a set is extracted as the common candidate product type set P _i = A _i ∩B _i .