JP5582610B2 - Image identification apparatus and program - Google Patents

Description

  The present invention relates to an image identification apparatus for identifying an input image by comparing the input image with a template image and selecting a template image that most closely approximates the input image, and in particular, a tablet with PTP (press through package) packaging. The present invention relates to an image identification device suitable for identifying drugs such as capsules.

  Dispensing error is a mistake in the medicine and amount of medicine prescribed by the pharmacist. Since a drug error that occurs at the time of delivery of a drug at a pharmacy is related to human life, there is a strong demand for its complete prevention measures at all times.

  By the way, many causes of drug errors are human errors such as misreading prescriptions and drug labels by pharmacists. Therefore, in order to prevent such a human error, it is very effective to introduce an identification device that mechanically identifies the type of drug and checks whether there is a drug error.

  As a conventional apparatus for checking a medicine error, those described in Patent Documents 1-3 are known.

  The tablet appearance inspection apparatus described in Patent Document 1 is an apparatus that performs an appearance inspection of a packaged tablet during PTP packaging. Many tablets and capsules prescribed by pharmacists are packed in PTP. PTP packaging refers to packaging in which tablets are placed in a transparent hard resin film or the like formed with pockets that are recessed in the form of tablets or capsules and sealed with an aluminum film or the like. The external appearance inspection apparatus described in Patent Document 1 is intended to inspect the external appearance of a tablet after the tablet is filled in a pocket portion of a hard resin film and before sealing with an aluminum film. The appearance inspection apparatus captures a tablet contained in a pocket portion to generate color image data, and among the color image data, a predetermined number of pixels from a plurality of pixels corresponding to the tablet in descending order of luminance. Is extracted, an average value of the luminances of a predetermined number of extracted pixels is obtained for each color component, and a threshold value is determined for each color component to detect a defective appearance of the tablet.

  The tablet identification device described in Patent Document 2 irradiates the surface of a tablet with a light source, receives the reflected light, and identifies the tablet based on the spectrum of the reflected light.

  The tablet identification device described in Patent Document 3 is such that identification information such as a barcode is attached in advance to a tablet packaging container, and tablet identification is performed by reading this identification information.

JP 2006-194801 A Japanese Patent Laid-Open No. 10-246702 JP-A-9-253164

  The appearance inspection apparatus of Patent Document 1 is for inspecting tablets just before PTP packaging of tablets. However, in actual pharmacies, tablets may be stored in a PTP-packed state, and tablets that have already been PTP-packed may be taken out and handed to the patient during prescription. Therefore, in this case, the identification inspection of the PTP-packed tablets must be performed.

  When the appearance inspection apparatus of Patent Document 1 is applied to the identification inspection of PTP-packed tablets, the luminance inspection is performed by imaging the tablets from above the transparent hard resin film. However, in this case, the brightness of the photographed tablet image changes greatly depending on the reflection state of the illumination light on the hard resin film. Therefore, it is difficult to uniquely determine the discrimination threshold, and it is difficult to accurately identify tablets with the appearance inspection apparatus of Patent Document 1.

  In addition, many tablets have similar colors and shapes, and these similar tablets need to be identified in a PTP packaged state. Therefore, as in Patent Document 2, it is difficult to accurately identify tablets having similar colors and shapes by the method of performing a spectrum inspection of reflected light.

  Further, as in the tablet identification device described in Patent Document 3, if identification information such as a barcode is attached to the PTP package in advance, it is possible to identify the tablet without making a mistake. However, in reality, not all tablets are attached with barcodes, and a dedicated PTP packaging container and a management system are separately required to realize this.

  Therefore, an object of the present invention is to enable tablet identification with high accuracy on the basis of image data of a PTP-packed tablet imaged by a normal imaging means during identification inspection of a PTP-packed tablet. An object of the present invention is to provide an image identification device and a program thereof.

  Although the present invention is mainly intended for identification inspection of PTP-packed tablets, the present invention can be applied to identification inspection of various articles other than PTP-packed tablets. .

An image identification apparatus according to the present invention includes a target image storage unit that stores a target image that is a target of identification;
Template storage means for storing color dispersion descriptors of a plurality of template images for identifying the target image;
Object image storage means for storing an image of an object image extracted from the target image;
Object image extraction means for reading the target image from the target image storage means, extracting an object image from the target image, and storing the object image in the object image storage means;
An average value vector p _ave of color vectors p _i (i = 1,..., N; N is the number of pixels of the object image image) of each pixel of the object image image stored in the object image storage means is expressed by the following equation (1c). Mean value vector calculation means calculated by:
Wherein the average value vector p _ave and color vector p _i of each pixel of the stored object image image on the object image storage means, and covariance matrix calculation means for calculating a variance-covariance matrix S by the following equation (1a),
Eigenvalue / eigenvector computing means for calculating eigenvalues λ _k and eigenvectors φ _k (k = 1,..., K; || φ _k || ² = 1) of the variance-covariance matrix S;
A CDD operation means for the by the following formula from the respective eigenvalue lambda _k variance-covariance matrix S (1d) to calculate the contribution rate r _k, calculates the chromatic dispersion descriptor D which is defined by the following equation (1e),
The color dispersion descriptor D _α (α = 1,..., M; M is the number of template images) of each template image stored in the template storage means, and the color dispersion descriptor D calculated by the CDD calculation means CDD distance calculating means for calculating the distance S _CV (D, D _α ) of the following equation (1f), (1g):
Among the color dispersion descriptors of each template image stored in the template storage means, the color dispersion descriptor of the template image having the smallest distance S _CV (D, D _α ) is selected, and the color dispersion of the selected template image is selected. Template selection means for outputting the descriptor number α as an identification result.

  As a result of the experiment, it has been found that tablets can be identified with extremely high accuracy based on image data of PTP-packed tablets photographed by ordinary photographing means.

  In the present invention, identification is performed using “color” which is an important characteristic of PTP packaging. Generally, PTP packaging contains tablets and capsules inside, and the surface thereof is printed with characters and marks. Each of these is color-coded and has a variety of colors for each PTP package. Focusing on this fact, in the present invention, color information is used as a feature quantity for identification. On the other hand, many descriptors related to color have been proposed. For example, in the moving image search of MPEG7, “Color Layout Descriptor (CLD)”, “Dominant Color Descriptor (DCD)”, and the like are used.

  The hue distribution descriptor is a descriptor that focuses on the hue value (hue) of each pixel in the image, and holds the appearance ratio as a histogram. Since objects of the same kind are considered to have similar colors to some extent, a desired pattern can be found by measuring the distance between histograms. However, the hue distribution descriptor has a problem that the feature cannot be extracted from a gray image having no hue. In particular, in the case of tablets mainly made of white and silver, such as PTP-packed tablets, captured images tend to be close to gray images (monotone images). Distribution descriptors are not suitable as features for identification.

  The representative color descriptor extracts several representative colors in the image and holds how much each representative color appears. A representative color descriptor can characterize an image well if it is deterministic that the image consists of several colors. However, the actual number of constituent colors of the image is not always a fixed number, and problems remain such as setting a good threshold and dealing with clustering failure in clustering when extracting representative colors.

Therefore, in the present invention, a color distribution descriptor (Color Distribution Descriptor: CDD) D defined by the equation (1e) is used as a color descriptor that can efficiently hold the color information included in the medicine package. Is newly devised and used as a feature value used for identification. The color dispersion descriptor D can be calculated at a very high speed. Unlike conventional methods, it also supports gray images and does not require any parameter settings or adjustments. Furthermore, since it is based on a statistical method called principal component analysis, it is not greatly affected by disturbances such as slight noise due to reflection of illumination light. In the color dispersion descriptor D, the size is always a fixed K ² element (9 elements in the case of an RGB image (K = 3)) and is very compact. It has been demonstrated that 120 packages can be processed per second, and the identification accuracy is 98%, which is demonstrated to be extremely high speed and high accuracy. Details thereof will be described later.

Further, in the present invention, the template selection means has a predetermined number of templates from the one having the smaller distance S _CV (D, D _α ) among the color dispersion descriptors of the template images stored in the template storage means. It is also possible to select the color dispersion descriptor of the image and output the number α of the color dispersion descriptor of the selected template image as the identification candidate result.

Thereby, when there are a plurality of candidates that the distance S _CV (D, D _α ) approximates, the plurality of candidates can be output as identification candidate results.

  The program of the present invention is characterized in that the computer is operated as the above-described image identification device by being read and executed by the computer.

  As described above, according to the present invention, the influence of noise due to the reflection of illumination light and the color of the image peculiar to PTP packaging are based on the image data of PTP-packed tablets photographed by a normal photographing means. Tablets can be identified with extremely high accuracy without being greatly affected by monotonicity (monotone properties). Accordingly, it is possible to effectively prevent a drug error due to a human error at a drug prescription site.

It is a figure showing the structure of the image identification device which concerns on Example 1 of this invention. It is a figure which shows an example of the image data of the sample of the tablet by which PTP packaging was carried out. Distance in equation (1 g) d is an example of _{(r K φ K, r α} , K φ α, K). It is a figure showing the result of having compared the recognition rate at the time of using distance _SCS , _SEV , _SCV . It is a figure showing the comparison of the cumulative recognition rate of CDD and SCD.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

[1] Configuration FIG. 1 is a diagram illustrating a configuration of an image identification device according to Embodiment 1 of the present invention. In FIG. 1, an image identification device 1 according to the present embodiment is a device for identifying the type of a tablet based on image data of a PTP-packed tablet imaged by an imaging device 2. The image identification device 1 includes a target image storage unit 3, a template storage unit 4, an object image storage unit 5, an object image extraction unit 6, an average value vector calculation unit 7, a covariance matrix calculation unit 8, an eigenvalue / eigenvector calculation unit 9, A CDD calculation means 10, a CDD distance calculation means 11, a template selection means 12, an output device 13, and a template storage means 14 are provided. These configurations may be configured in hardware using a programmable logic device such as a dedicated LSI or general-purpose FPGA, but are configured by software as a program, loaded into a computer, and executed. The image identification device 1 of FIG. 1 may be functionally realized.

  The target image storage unit 3 stores image data captured by the imaging device 2. In the imaging device 2, a target image that is an image of a PTP-packed tablet that is an identification target is captured. This target image is stored in the target image storage means 3.

The template storage unit 4 stores information on M (M> 1) template images for identifying the target image. The template image is an image obtained by photographing samples of various tablets packed with PTP. Here, what is stored in the template storage unit 4 is not the template image itself but the color dispersion descriptor D calculated from the template image. In the following, alpha-th (α = 1, ..., M ) color dispersion descriptor D of the template image referred to as D _alpha.

  The object image storage unit 5 stores an image of an object image extracted from the target image.

  The object image extraction unit 6 reads out the target image from the target image storage unit 3, extracts the object image from the target image, and stores it in the object image storage unit 5 as the object image image. Various methods such as region extraction by edge detection are already known as methods for extracting an object image from a target image, and object images are extracted by these known methods.

The average value vector calculation means 7 is an average value vector of the color vectors p _i (i = 1,..., N; N is the number of pixels of the object image image) of each pixel of the object image image stored in the object image storage means 5. p _ave is calculated by the equation (1c).

The covariance matrix calculation means 8 calculates the average value vector p _ave and the color vector p _i (i = 1,..., N) of each pixel of the object image stored in the object image storage means 5 from the equation (1a). A variance-covariance matrix S is calculated. The color vector p _i is a vector having the luminance value of each color component of the pixel as its element, and has the same dimension as the number of colors K of the pixel. In the case of a normal RGB image, K = 3, but in the case of a CMYK image, K = 4, and in the case of a gray image, K = 1.

The eigenvalue / eigenvector computing means 9 calculates the eigenvalue λ _k and eigenvector φ _k (k = 1,..., K; || φ _k || ² = 1) of the variance-covariance matrix S. A known method is used to calculate the eigenvalue λ _k and the eigenvector φ _k .

CDD calculating means 10, each eigenvalue lambda _k variance-covariance matrix S (k = 1, ..., K) to calculate the contribution rate _{r k} from the equation (1d), chromatic dispersion descriptions that are defined by the formula (1e) Child D is calculated.

The CDD distance calculation means 11 is a color dispersion descriptor D _α (α = 1,..., M; M is the number of template images) of each template image stored in the template storage means 4 and the color calculated by the CDD calculation means 10. The distance S _CV (D, D _α ) with the dispersion descriptor D is calculated by the equations (1f) and (1g).

The template selection unit 12 selects a template image having the smallest distance S _CV (D, D _α ) from among the template images stored in the template storage unit 4 and outputs the selected template image number α as an identification result. To do. In this embodiment, the template selection unit 12 outputs only the number of the template image having the smallest distance S _CV (D, D _α ). However, if necessary, the distance S _CV (D, D _A predetermined number of template images may be selected from the smaller _{α 1} ), and the number α of the selected template images may be output as an identification candidate result.

  The output device 13 is an output device including a display, a printer, an external storage device, and the like, and displays, prints, stores, and the like the identification results and identification candidate results output from the template selection unit 12.

The template storage means 14 is based on image data obtained by photographing various tablet samples, the object image extraction means 6, the average value vector calculation means 7, the covariance matrix calculation means 8, the eigenvalue / eigenvector calculation means 9, and the CDD calculation. The color dispersion descriptor D _α of each template image generated by the means 10 is stored in the template storage means 4.

[2] Operation The operation of the image identification apparatus 1 according to the present embodiment configured as described above will be described below. In the image identification device 1, as advance preparations, samples of various tablets packed in PTP are photographed, and the color dispersion descriptors D _α (α = 1,..., M) of each template image are stored in the template storage unit 4. Save it. After that, the PTP-packed tablets that are actually inspected are identified.

(1) Template information creation process (1-1) First, the imaging device 2 images a sample of a PTP-packed tablet, and the image data (target image) of the sample is stored in the target image storage means 3. FIG. 2 shows an example of image data of a sample of a PTP-packed tablet.

  (1-2) Next, the object image extraction unit 6 cuts out the sample object image from the sample image data, and the extracted sample object image is stored in the object image storage unit 5.

(1-3) Next, the average value vector calculator 7 calculates the color vector p _i (i = 1,..., N; N is the object image image) of each pixel of the object image stored in the object image storage 5. The average value vector p _ave of the number of pixels) is calculated by equation (1c).

(1-4) Next, the covariance matrix calculator 8 calculates the average value vector p _ave and the color vector p _i (i = 1,..., N) of each pixel of the object image stored in the object image storage 5. ) To calculate the variance-covariance matrix S according to the equation (1a). Here, the variance-covariance matrix S is a square matrix having K × K elements. When the target image to be captured first is an RGB image, K = 3, and the variance-covariance matrix S is a square matrix of 3 × 3 elements.

(1-5) Next, the eigenvalue / eigenvector computing means 9 calculates the eigenvalue λ _k and eigenvector φ _k (k = 1,..., K) of the variance-covariance matrix S.

(1-6) The CDD calculation means 10 calculates the contribution rate r _k from each eigenvalue λ _k of the variance-covariance matrix S by the equation (1d), and calculates the chromatic dispersion descriptor D defined by the equation (1e). To do.
Here, K eigenvalues λ _k and eigenvectors φ _k are respectively calculated. The eigenvector φ _k has a constant multiple of degrees of freedom, but here the eigenvector φ _k is calculated so as to satisfy the normalization condition || φ _k || ² = 1. Each eigenvector phi _k represents a unit vector pointing respectively to the three principal axis directions of distribution of the color vector p _i. Further, the eigenvalue λ _k represents the dispersion along the k-th principal axis. Therefore, the formula (1d), which represents the contribution ratio _{r k} is the ratio of the variance of the distribution of the main axis of the color vector _{p i} in (1e). For convenience of explanation, it is assumed that λ ₁ ≧ λ ₂ ≧ λ ₃ ≧ 0 and 0 ≦ r ₃ ≦ r ₂ ≦ r ₁ ≦ 1.

If PTP-packed tablets are almost similar to grayscale images (for example, when white tablets are enclosed on a silver sheet and printed black), the main axis φ ₁ plays a dominant role. As a result, the other main axes φ ₂ and φ ₃ are almost meaningless. In this case, r ₁ ≈1, r ₂ ≈0, and r ₃ ≈0. Thus, the contribution rate r _k of the formula (1d) can be seen to be suitable for use as a weighting factor. Hereinafter, the vector r _k φ _k is referred to as “contribution vector”. Chromatic dispersion descriptor D is a set of K contribution vector r _k phi _k of the distribution of the color vector p _i.

(1-7) Finally, the template storage unit 14 uses the αth (α = 1,..., M; M is the number of template images) template image as the chromatic dispersion descriptor D calculated by the CDD calculation unit 10. as chromatic dispersion descriptor D _alpha of, stored in the template storage unit 4.

The above operations (1-1) to (1-7) are repeatedly performed on a sample of M PTP-packed tablets. As a result, the template storage means 4 stores the color dispersion descriptors D _α (α = 1,..., M) of M sets of template images.

(2) Tablet Identification Processing After preparation in advance is performed by performing the template information creation processing of (1) above, next, PTP-packed tablets are identified by the following tablet identification processing.

  (2-1) First, the imaging device 2 images a PTP-packed tablet that is an identification target, and stores the image data in the target image storage unit 3 as a target image.

(2-2) Next, the color dispersion descriptor D of the target image D = {r ₁ φ ₁ ,..., R _K φ _K } by the same processing operation as the above (1-2) to (1-6). Is calculated.

(2-3) Next, the CDD distance calculation means 11 determines the color dispersion descriptor D _α (α = 1,..., M; M is the number of template images) of each template image stored in the template storage means 4. The distance S _CV (D, D _α ) from the chromatic dispersion descriptor D calculated by the CDD calculation means 10 is calculated by the following equations (1f) and (1g).

The distance S _CV (D, D _α ) between the color dispersion descriptors D and D _α is defined by the equations (1f) and (1g). Here, || · || represents the L1 norm, and the L1 norm of the vector x = [x ₁ ,..., X _K ] is calculated by || x || = | x ₁ | + ... + | x _K | Is done. FIG. 3 illustrates an example of the distance d (r _k φ _k , r _{α, k} φ _{α, k} ) in the equation (1g). Since indefiniteness remains regarding the sign of the eigenvector φ _k, the sign of the eigenvector φ _k is defined as in equation (1g) so that the distance is the same whether positive or negative.

(2-4) Finally, the template selection unit 12 selects a template image having the smallest distance S _CV (D, D _α ) from the template images stored in the template storage unit 4 and selects the selected template image. Is output as the identification result. The output device 13 outputs the identification result output from the template selection unit 12 by display, printing, or the like.

[3] Evaluation of operation First, in order to evaluate the validity of the expression (1f) which is the definition of the distance S _CV (D, D _α ), other definition of the distance S _EV (D, D _α ), S _CS Comparison with (D, D _α ) was performed. The distance S _EV (D, D _α ) is set to r _k = 1 in the formula (1f) and is defined by the following formula (2).

The distance S _CS (D, D _α ) is a cosine similarity and is defined by the following equation (3).

For the evaluation, the various PTP-packed tablets shown in FIG. 2 were used, and for each type, a plurality of image data obtained by photographing the PTP-packed tablets were used, and the calculation results of the identification rates were compared. The number of colors was evaluated for K = 1, 2, 3 respectively. FIG. 4 shows recognition rates when distances S _CS , S _EV , and S _CV are used. As is clear from FIG. 4, the highest recognition rate was obtained when the distance _SCV was used, and the recognition rate was about 98.7% when K = 3. From this, it was confirmed that the use of distance S _CV defining an appropriate formula (1f).

  Next, the recognition rate was compared with a scalable color descriptor (SCD) used in MPEG7. FIG. 5 shows the result. The horizontal axis in FIG. 5 is the number of selections indicating the number of similar template images selected from the smaller distances as the identification candidate result, and the vertical axis represents the ratio of the correct template images included in the identification candidates. Represent. As can be seen from FIG. 5, when the scalable color descriptor (CSD) is used, the recognition rate is about 89% when the number of selections is 1, whereas the number of selections is about the color dispersion descriptor (CDD). When the value is 1, the recognition rate is about 98.7%, and it can be seen that CDD provides an overwhelmingly superior result.

DESCRIPTION OF SYMBOLS 1 Image identification device 2 Imaging device 3 Target image storage means 4 Template storage means 5 Object image storage means 6 Object image extraction means 7 Average value vector calculation means 8 Covariance matrix calculation means 9 Eigenvalue / eigenvector calculation means 10 CDD calculation means 11 CDD Distance calculation means 12 Template selection means 13 Output device 14 Template storage means

Claims (3)

Target image storage means for storing a target image that is a target of identification;
Template storage means for storing color dispersion descriptors of a plurality of template images for identifying the target image;
Object image storage means for storing an image of an object image extracted from the target image;
Object image extraction means for reading out the target image from the target image storage means, extracting an object image from the target image, and storing the object image in the object image storage means;
An average value vector p _ave of color vectors p _i (i = 1,..., N; N is the number of pixels of the object image image) of each pixel of the object image image stored in the object image storage means is expressed by the following equation (1c). Mean value vector calculation means calculated by:
Wherein the average value vector p _ave and color vector p _i of each pixel of the stored object image image on the object image storage means, and covariance matrix calculation means for calculating a variance-covariance matrix S by the following equation (1a),
Eigenvalue / eigenvector computing means for calculating eigenvalues λ _k and eigenvectors φ _k (k = 1,..., K; || φ _k || ² = 1) of the variance-covariance matrix S;
A CDD operation means for the by the following formula from the respective eigenvalue lambda _k variance-covariance matrix S (1d) to calculate the contribution rate r _k, calculates the chromatic dispersion descriptor D which is defined by the following equation (1e),
The color dispersion descriptor D _α (α = 1,..., M; M is the number of template images) of each template image stored in the template storage means, and the color dispersion descriptor D calculated by the CDD calculation means CDD distance calculating means for calculating the distance S _CV (D, D _α ) of the following equation (1f), (1g):
Among the color dispersion descriptors of each template image stored in the template storage means, the color dispersion descriptor of the template image having the smallest distance S _CV (D, D _α ) is selected, and the color dispersion of the selected template image is selected. An image identification device comprising: template selection means for outputting a descriptor number α as an identification result.
The template selecting unit, wherein among the chromatic dispersion descriptor for each template image stored in the template storage unit, the distance S _{CV (D, D α)} chromatic dispersion descriptor of a predetermined number of the template image from the smaller The image identification apparatus according to claim 1, wherein the number α of the color dispersion descriptor of the selected template image is output as an identification candidate result.
  A program that causes a computer to operate as the image identification device according to claim 1 by being read and executed by the computer.