JP7310045B1 - Single-package inspection device, single-package inspection program, and single-package inspection method - Google Patents

Abstract

Problem: To provide a package inspection device, a package inspection program, and a package inspection method that can inspect an unregistered drug even when drug data of the unregistered drug is not registered in a master table. Solution: Imaging devices 31 and 32 that image the sachet 21, a storage device 34 that stores drug master information, and an inspection processing device 33 that determines the suitability of the medicine sealed in the sachet 21. The inspection processing device 33 includes an acquisition means for acquiring prescription data and drug master information, a normal inspection means for determining suitability of drugs based on the prescription data and drug master information, and drug data of drugs included in the prescription. a drug data temporary registration means for creating temporary drug master information of the unregistered drug from the captured images taken by the imaging devices 31 and 32 when the drug is an unregistered drug that is not registered as drug master information; Temporary master inspection means is provided for determining suitability of unregistered drugs based on master information.

Description

TECHNICAL FIELD The present invention relates to a package inspection device, a package inspection program, and a package inspection method for inspecting whether a drug is packaged in conformity with a prescription. The package inspection program according to the present invention is software that can be executed on a computer, and can be stored in computer-readable storage media such as CD-ROMs, DVDs, USB memories, HDDs, and SSDs (Solid State Drives). is.
In this specification, the term "single packaging inspection" refers to a process called "single packaging" in which the drug described in the prescription is packaged for each dose, and the packaged drug is It means to check whether the medicine conforms to the prescription and whether the quantity conforms to the prescription.

In pharmacies, a process called single packaging is performed in which a drug described in a doctor's prescription is packaged for each dose. However, when the medicines are packed into one package, there are cases where more or less medicine than the prescribed quantity is mistakenly enclosed, or different medicines are enclosed. Therefore, conventionally, a camera is used to capture an image of a drug packaged in a sachet, analyze the characteristics of the drug based on the image information of the captured drug, and use the analysis results to identify the packaged drug. A device for inspecting whether a prescription is conformed is known (for example, Patent Literature 1).

JP 2018-117921 A

For example, in Patent Literature 1, the drug enclosed in the sachet is specified by comparing the analysis result of the imaged drug with the drug data stored in advance in a storage table. If the medicine data of the enclosed medicine is not registered in advance in the storage table, there is a problem that this medicine cannot be specified. In particular, even when the drug data in the storage table is updated periodically (for example, once a day), the information in the storage table is updated by downloading the drug data from the external server. Until the drug data is registered, the unregistered drug cannot be registered even in the storage table of the package inspection device, and the unregistered drug cannot be identified for a certain period of time (long term). there were.

The present invention provides a package inspection device, a package inspection program, and a package inspection method that can inspect an unregistered drug even when drug data of the unregistered drug is not registered in a master table. intended to

A package inspection device according to the present invention is a package inspection device for inspecting whether a medicine packaged in each package bag of a series of connected package bags conforms to a prescription. An imaging device that captures an image of a package, a storage device that stores drug master information, and an inspection processing device that determines compatibility of the drug enclosed in the sachet, wherein the inspection processing device stores a prescription. acquisition means for acquiring data and drug master information; normal inspection means for determining suitability of the drug based on the prescription data and the drug master information; and drug data of the drug contained in the prescription being the drug master information. drug data temporary registration means for creating drug temporary master information of the unregistered drug from the captured image captured by the imaging device when the drug is not registered as a drug, based on the drug temporary master information and a provisional master inspection means for determining suitability of the unregistered drug, wherein the drug data provisional registration means calculates a feature amount of the unregistered drug from the captured image using a machine learning model. Grouping is performed to classify the unregistered drugs into groups based on the feature amounts of the unregistered drugs obtained, and the temporary drug master information is created for each of the unregistered drugs classified into the same group. I can .
In the single package inspection device, the drug data temporary registration means has a predetermined inner product of a vector corresponding to the feature quantity of one unregistered drug and a vector corresponding to the feature quantity of the other unregistered drug. is equal to or less than the reference value, the one unregistered drug and the other unregistered drug can be classified into the same group.
In the single-package inspection device, the drug data temporary registration means provides images of the plurality of unregistered drugs classified into the same group, based on the characteristic amounts of the plurality of unregistered drugs classified into the same group. A representative image is selected from the above, and the drug temporary master information is created based on the selected representative image.
In the single-package inspection device, the drug data temporary registration means obtains a median value of characteristic amounts of a plurality of unregistered drugs classified into the same group, and obtains an image of the plurality of unregistered drugs classified into the same group. The image of the unregistered drug having the feature value closest to the median value may be selected as the representative image.
In the single package inspection device, the drug data provisional registration means has a function of determining the shape of the unregistered drug using a machine learning model for shape determination, and the unregistered drug is capable of extracting character information. If the shape is unnecessary, the image of the unregistered drug may be smoothed before the grouping.
In the single package inspection apparatus, the drug data provisional registration means, when the unregistered drug has a shape requiring extraction of character information, extracts each character from the image of the unregistered drug in the captured image. A function to detect areas where information exists using a machine learning model for detecting character areas, and a function to identify each character in the area where the detected character information exists using a machine learning model for character identification. , can be configured to include.
In the single-package inspection apparatus, the drug data temporary registration means performs edge amount calculation processing on the image of the unregistered drug, and the unregistered drug whose calculated edge amount exceeds a predetermined value is registered as the unregistered drug. It can be configured to be excluded from the drug group.
A package inspection program (or a storage medium recording the package inspection program) according to the present invention is configured such that a computer stores a series of connected sachet packages in which a drug packaged in each packaging bag conforms to a prescription. a package inspection program (or a storage medium recording the package inspection program) for executing an inspection of whether to and normal inspection means for determining suitability of the drug based on the drug master information, and drug data of the drug included in the prescription based on the prescription data and the drug master information are registered in the drug master information. drug data provisional registration means for creating and registering drug provisional master information of the unregistered drug from the captured image taken by the imaging device when the drug is not registered, based on the drug provisional master information, Functioning as temporary master inspection means for judging suitability of the unregistered medicine, the medicine data temporary registration means uses a machine learning model from the captured image captured by the imaging device to use the feature amount of the unregistered medicine. is calculated, and based on the calculated feature amount of the unregistered drug, grouping is performed to classify the unregistered drug into groups, and the temporary drug master information is generated for each of the unregistered drugs classified into the same group. Create .
The single-packaging inspection method according to the present invention is a single-packaging inspection method for inspecting whether or not a drug packaged in each packaging bag of a series of connected sachets conforms to a prescription. an acquisition step of acquiring prescription data and drug master information, a normal inspection step of determining suitability of the drug based on the prescription data and the drug master information, and a drug sealed in the sachet by the drug master a drug data provisional registration step of creating and registering drug provisional master information of the unregistered drug from a captured image of the unregistered drug when the drug is an unregistered drug that is not registered as information; a provisional master inspection step of determining suitability of the unregistered drug based on the above; and calculating a feature amount of the unregistered drug from the captured image using a machine learning model in the drug data provisional registration step. performing grouping for classifying the unregistered drugs into groups based on the calculated feature amount of the unregistered drugs, and creating the drug temporary master information for each of the unregistered drugs classified into the same group.

According to the present invention, in the single-package inspection device, it is possible to create drug data of an unregistered drug that is not registered in the master table and provisionally register it as drug temporary master information. Until the drug data is registered, the unregistered drug can be inspected using the provisionally registered drug temporary master information of the unregistered drug.

1 is a configuration diagram of a single package inspection system according to the present embodiment; FIG. FIG. 4 is a diagram for explaining a series of sachets; 1 is a schematic configuration diagram of a package inspection device according to the present embodiment; FIG. It is a scene example for explaining the drug data temporary registration function according to the present embodiment. It is a figure for demonstrating the grouping process which concerns on this embodiment. FIG. 10 is a diagram for explaining the tendency of drug feature amounts; FIG. 10 is a diagram for explaining a method of calculating the degree of similarity between drugs; FIG. 10 is a diagram for explaining a method of selecting a representative image; FIG. It is a figure for demonstrating the preparation method of character symbol information. It is a figure which shows an example of the temporary master table which concerns on this embodiment. It is a figure which shows an example of the temporary master sub-table which concerns on this embodiment. It is a figure which shows the example of a screen which a display displays in this embodiment. 6 is a flow chart showing a package inspection process according to the present embodiment; 4 is a flow chart showing drug data temporary registration processing according to the present embodiment.

The present invention will be explained on the basis of the figures. In the following, as an embodiment of the package inspection device according to the present invention, a package inspection device that inspects packaged drugs and displays a display according to the inspection result, and a package inspection device having the same, are described. An inspection system will be described as an example. Moreover, in the present invention, the "medicine" includes at least tablets and capsules, and may be configured to exclude powders, liquids, and granules. Furthermore, the single package inspection apparatus according to the present invention inspects whether or not the packaged medicine "conforms to the prescription". It means to determine whether or not a required quantity of prescribed medicines (part or all of the medicines) among the listed medicines is contained in the sachet.

FIG. 1 is a configuration diagram of a package inspection system 1 according to this embodiment. As shown in FIG. 1, the package inspection system 1 according to this embodiment includes a receipt computer 10, a packing machine 20, a package inspection device 30, and an external server 40. FIG. Note that, as shown in FIG. 1, the receipt computer 10, the packaging machine 20, and the single-package inspection device 30 are installed in the pharmacy 200, and the external server 40 is a server of a system management company, a cloud, or the like. installed outside. In addition, the receipt computer 10, the packaging machine 20, and the package inspection device 30 can communicate with each other by wire or wirelessly, and the external server 40 communicates with the package inspection device 30 via the Internet line 3. It is possible to communicate with In FIG. 1, arrows indicate the flow of information or data.

Prescription information prescribed by a doctor is input to the receipt computer 10 either manually or automatically. Prescription information includes information such as drug names, quantities, and dosing methods. Based on the prescription information input to the receipt computer 10, prescription data in a format that can be used by the packaging machine 20 and the single-package inspection device 30 is created. The prescription data is stored in the receipt computer 10 , and the packaging machine 20 and the single-package inspection device 30 acquire the prescription data in the receipt computer 10 .

Based on the prescription data acquired from the receipt computer 10, the packaging machine 20 performs a process of enclosing (single packaging) the drug according to the prescription. The packaging machine 20 also packages the medicine for the number of doses according to the prescription data. Specifically, the packaging machine 20 pulls out a band of packaging paper prepared in advance, puts various medicines into the compartmentalized space of the packaging paper, and encloses them, thereby creating the packaging bag 21. can be done. In addition, since the sachet 21 is created by partitioning a band of sachet paper, as shown in FIG. A bag 21 is created. A plurality of connected sachets 21 are hereinafter referred to as a series of sachets 2 .

FIG. 3 is a schematic configuration diagram of the package inspection device 30. As shown in FIG. The packaging inspection device 30 has a pair of cameras 31 and 32, an inspection processing section 33, a storage section 34, a display 35, and a conveying section 36, as shown in FIG. In addition to storing a program for executing the package inspection process according to the present embodiment, the storage unit 34 also stores a master table 37 for storing drug data downloaded from the external server 40 and a It has a temporary master table 38 and a temporary master sub-table 39 for storing the drug data obtained. Note that the arrangement of each component in FIG. 3 is an example, and is not limited to the example shown in FIG. 3, for convenience of explanation, a pair of cameras 31 and 32, an inspection processing unit 33, a storage unit 34 (a master table 37, a temporary master table 38 and a temporary master sub-table 39) and the transport section 36 are illustrated by dashed lines, and the flow of signals between each component is illustrated by dashed lines.

As shown in FIG. 3, the pair of cameras 31 and 32 are arranged above and below the position where the series of sachets 2 are conveyed by the conveying section 36, respectively. The pair of cameras 31 and 32 captures images of both sides of each packaging bag 21 and transmits captured image data of the captured packaging bag 21 to the inspection processing unit 33 . The conveying unit 36 is not particularly limited as long as it conveys a series of sachets 2, and examples thereof include a belt conveyer, a linear conveyer, and a roller conveyer. In the case of a roller conveyor, a sandwich-type roller conveyor is preferable, in which a series of sachets 2 are conveyed while being sandwiched between two rollers arranged in the vertical direction.

The inspection processing unit 33 is, for example, a computer having a CPU, a RAM, and a ROM, and executes a package inspection program stored in the storage unit 34 to perform a prescription data acquisition function, an image data acquisition function, It executes an inspection function, a drug data temporary registration function, and an inspection result display function. Each function of the inspection processing unit 33 will be described below.

The prescription data acquisition function acquires prescription data input by the receipt computer 10 from the receipt computer 10 . Also, the image data acquisition function acquires captured image data of each sachet 21 captured from the pair of cameras 31 and 32 . The prescription data acquisition function can also be configured to acquire the prescription data from the receipt computer 10 or the packaging machine 20 after the packaging machine 20 acquires the prescription data from the receipt computer 10 and performs packaging.

The inspection function determines whether the drug enclosed in the sachet 21 is compatible with the prescription based on the prescription data acquired by the prescription data acquisition function and the captured image data acquired by the image data acquisition function. to inspect. Specifically, the inspection function first identifies the type and number of medicines to be enclosed in the sachet 21 based on the prescription data. The inspection function also refers to the drug master table 37, the temporary master table 38, and the temporary master sub-table 39 stored in the storage unit 34, and acquires the drug data of the prescription drug. Drug data includes drug form (tablet or capsule), drug shape (outer shape or contour such as round or oval), size, color, and marking/printing on the tablet (characters, patterns such as pictures, symbols, or stripes).

Furthermore, the inspection function determines whether or not there is a required amount of prescription drugs in the captured image based on the captured image data of the sachet 21, the prescription data, and the drug data, thereby encapsulating it in the sachet 21. Check whether the prescribed drug complies with the prescription. Specifically, the inspection function first identifies the drug to be enclosed in the sachet 21 from the prescription data. Next, the inspection function determines, for each drug registered in the prescription data, whether or not the drug exists in the required quantity (the quantity registered in the prescription data) in the captured image. For example, the inspection function compares the drug in the captured image with the drug data of the drug in the prescription, and the drug in the captured image must match the form, shape, size, color, and engraving/printing of the drug in the prescription. If the quantity is present, the drug is determined to "match" the prescription. The inspection function can also be configured to identify the drug in the captured image that corresponds to the prescription drug by image analysis. It is also possible to identify the drug in the captured image. Note that, in the present embodiment, the package inspection device 30 downloads update data from the external server 40 at a predetermined cycle (for example, once a day), so that the drug master table 37 stored in the storage unit 34 can be updated. is updated.

In addition, in the present embodiment, the inspection function cannot determine that the drug registered in the prescription data exists in the captured image, and if it is determined to be "nonconforming", the drug is , a first non-conforming condition or a second non-conforming condition. Here, as a first non-conforming condition, for example, the case where the drug registered in the prescription data has no engraving/printing is exemplified. If there is no engraving/printing on the medicine, it may not be possible to judge the medicine with high accuracy.

In addition, the inspection function cannot determine the drug if the drug registered in the prescription data is not registered in the master table 37 and the temporary master table 38, so that the inspection function satisfies the second non-conformity condition "nonconforming". ”. Furthermore, if the medicine registered in the prescription data does not satisfy the first incompatibility condition, but the medicine cannot be identified in the captured image, the inspection function satisfies the second incompatibility condition. ” is determined. For example, in the captured image, only the side of the medicine is imaged (the plane of the medicine cannot be imaged), and the shape, engraving, printing, etc. of the medicine cannot be identified, and the prescription medicine cannot be identified. judges the drug in the prescription as "incompatible" that satisfies the second incompatibility condition.

The drug data provisional registration function creates drug data for the drug when the drug data of the drug contained in the sachet 21 is not stored in the master table 37, and temporarily stores the created drug data in the storage unit 34. It is temporarily registered in the master table 38 and the temporary master sub-table 39 . Specifically, when the inspection function cannot identify the drug in the sachet 21 even when the inspection function refers to the master table 37, the drug data provisional registration function creates drug data based on the captured image of the drug. The created drug data is provisionally registered in the provisional master table 38 and provisional master sub-table 39 . In the example shown in FIG. 4, drug data for drugs A to E are registered in the master table 37, and drug data for drugs X and Y are not registered in the master table 37 (and temporary master table 38). A case is illustrated and demonstrated. Further, in the following description, medicines that are not registered in the master table 37 and the temporary master table 38 will be described as unregistered medicines.

When temporarily registering the drug data of an unregistered drug, the drug data provisional registration function first cuts out images of all the drugs enclosed in the sachet 21 from the captured image of the sachet 21 . Specifically, the drug data provisional registration function uses a machine learning model M1 for determining areas in which drugs are present, and extracts each drug A to Areas where E, X, and Y are present are detected, and images of areas where each drug A to E, X, and Y are present are cut out. At this stage, the images of drugs A to E registered in master table 37 or temporary master table 38 and the images of unregistered drugs X and Y not registered in master table 37 or temporary master table 38 are also cut out. will be Further, in the present embodiment, the medicine data temporary registration function cuts out images of medicines A to E, X, and Y from a plurality of (for example, three or more) captured images of the sachet 21 .

Then, the medicine data temporary registration function performs a process of grouping images of unregistered medicines that are not registered in the master table 37 and the temporary master table 38 based on the clipped medicine images. For example, in the example shown in FIG. 5, FIG. 5(Z) shows images of drugs A to E, X, and Y cut out from the captured image, and the drug data temporary registration function is the image of FIG. 5(Z). Among them, the images of medicines A to E already registered in the master table 37 are excluded from the target of temporary registration (see FIG. 5 (Z')). Then, as shown in FIGS. 5(X) and 5(Y), the drug data temporary registration function classifies the unregistered drugs X and Y, which are not registered in the master table 37 and the temporary master table 38, into separate groups. Thus, grouping processing of unregistered medicines is performed. FIG. 5 is a diagram for explaining the grouping process according to this embodiment. The details of the grouping process by the drug data temporary registration function will be described below.

In this embodiment, the drug data provisional registration function first calculates the feature amount of each drug using the machine learning model M2 for calculating the feature amount from the image of each drug cut out from the captured image. For example, in the present embodiment, the drug data provisional registration function can use the machine learning model M2 to calculate 512 feature amounts as the feature amounts of the drug from one drug image.

Next, the drug data provisional registration function excludes drug images that have already been registered in the master table 37 and the provisional master table 38 from among the clipped drug images from the target of provisional registration. Here, in the master table 37 and the temporary master table 38, feature amounts of already registered medicines are stored in advance (see FIG. 10, which will be described later), and the medicine data temporary registration function is performed by each Based on the feature amount of each drug based on the image of the drug and the feature amount of the drug already registered in the master table 37 and the temporary master table 38, a master table is selected from the image of each drug cut out from the captured image. 37 or the image of the drug already registered in the temporary master table 38 is specified. Specifically, the drug data temporary registration function refers to the prescription data, the master table 37 and the temporary master table 38, and registers the drug described in the prescription in the master table 37 or the temporary master table 38. obtain drug data of registered drugs (hereinafter also referred to as registered drugs). It should be noted that the drug data of the registered drug includes information on the feature amount of the registered drug. Then, the drug data provisional registration function obtains the degree of similarity between each drug extracted from the captured image and the registered drug based on the feature amount of each drug extracted from the captured image and the feature amount of the registered drug, Among the medicines cut out from the captured image, medicines with a degree of similarity equal to or higher than a predetermined reference value s are regarded as medicines registered in the master table 37 or the temporary master table 38, and excluded from temporary registration targets. Details of how to obtain the degree of similarity will be described below.

For example, when the feature values of a plurality of types of drugs are converted into coordinates in a coordinate space of dimensions corresponding to the number of feature values (for example, a 512-dimensional coordinate space when calculating 512 feature values), usually, Similar feature values are obtained for drugs of the same type, so they are arranged at relatively close positions on the coordinate space. For example, FIG. 6 is a diagram for explaining the trend of drug feature amounts. Two feature amounts are calculated for each of the drugs A to E, X, and Y, and the calculated feature amounts are stored in a two-dimensional coordinate space. Schematic diagram plotted above. When feature amounts are calculated for drugs A to E, X, and Y, as shown in FIG. 6, for drugs of the same type, the coordinates corresponding to the feature amounts tend to be located close to each other on the coordinate space. In addition, when the coordinates corresponding to the feature amounts of the two drugs to be compared are located close to each other on the coordinate space, the vectors corresponding to the feature amounts are also approximated (for example, in the example shown in FIG. 6, the same kind of drug The vector x1 and the vector x2 of X approximate, but the vector x1 of the different drug X and the vector y of the drug Y do not approximate). is calculated as the degree of similarity, drugs already registered in the master table 37 or the temporary master table 38 can be identified from among the drugs cut out from the captured image.

FIG. 7 is a diagram for explaining a method of calculating the degree of similarity between drugs according to this embodiment. In the example shown in FIG. 7, a two-dimensional coordinate space in the case of calculating feature amounts of two medicines to be compared will be described as an example. By doing so, the degree of similarity can be similarly calculated.

For example, in the example shown in FIG. 7, when obtaining the degree of similarity between drug a and drug b, the drug data provisional registration function first performs vector a corresponding to the feature amount of drug a and vector a corresponding to the feature amount of drug b. and vector b are calculated. In this embodiment, the drug data provisional registration function is used to determine the degree of similarity between drug a and drug b using the directions of vector a and vector b (or the angle θ formed by vector a and vector b) as a criterion for determination. , normalize the magnitudes of vector a and vector b to '1', without needing the magnitudes of vector a and vector b. In FIG. 7, vector a' indicates the vector of drug a before normalization, and vector a indicates the vector of drug a after normalization. Then, the drug data provisional registration function obtains the inner product of vector a and vector b using the following formula. The vector a has _vector values _{a 1} , _{a 2 ,} . • Let b have _n vector values.
As shown in the above formula, the inner product of vector a and vector b can be obtained as the sum of the products of the respective vector values.

Here, when the two drugs a and b have exactly the same feature amount, the inner product of vector a and vector b obtained by the above equation is 1. Therefore, the drug data provisional registration function can determine that two drugs are similar when the vector inner product calculated by the above formula is equal to or greater than a predetermined reference value s (0<s<1). On the other hand, if the result of calculating the degree of similarity between the two drugs to be compared is that the vector inner product is less than the predetermined reference value s, it can be determined that the two drugs are not similar. Note that the reference value s for judging the degree of similarity can be obtained, for example, by preparing drugs of the same type in advance and obtaining the inner product of vectors corresponding to the characteristic amounts of these drugs in advance. It is preferable to determine in advance a reference value s that is used as a reference for judging.

In this manner, the drug data temporary registration function calculates the degree of similarity between each drug cut out from the captured image and the drug already registered in the master table 37 or the temporary master table 38, and the calculated similarity is the reference value. s or more, it is possible to identify drugs already registered in the master table 37 or the temporary master table 38 from among the drugs cut out from the captured image, and exclude them from the target of temporary registration. . As a result, the drug data provisional registration function identifies drugs that have not already been registered in the master table 37 or the temporary master table 38 among the drugs cut out from the captured image as unregistered drugs to be provisionally registered. be able to.

Next, the drug data provisional registration function executes grouping processing for grouping images of unregistered drugs. This is for provisionally registering drug data for each type of unregistered drug when there are two or more types of unregistered drugs. Specifically, the drug data provisional registration function calculates the degree of similarity between unregistered drugs to be provisionally registered in order to classify images of drugs of the same type into the same group. Note that the degree of similarity can be obtained in the same manner as the method shown in FIG. 7 described above. Then, the drug data provisional registration function is such that when the inner product of vectors of two unregistered drugs to be compared is equal to or greater than a predetermined reference value s (0<s<1) and the two unregistered drugs are similar, Classify two unregistered drugs into the same group. On the other hand, the drug data provisional registration function calculates the degree of similarity between two unregistered drugs. into different groups. Thus, for example, in the example shown in FIG. 5, the drug data provisional registration function can classify the unregistered drugs X and Y into the unregistered drug X group and the unregistered drug Y group.

The drug data provisional registration function can also execute the following processing when grouping unregistered drugs. That is, the drug data provisional registration function uses the machine learning model M1 to execute the shape determination logic for determining the silhouette (shape) of the image of the unregistered drug. Determine if it is an ellipse, capsule, etc. If the shape of the unregistered drug is a capsule or a semicircle, the drug data provisional registration function uses a smoothing filter to perform grouping without recognizing the stamped or printed character symbols. is used to smooth the image, and then the feature amount is calculated for the smoothed image, whereby grouping can be performed.

Furthermore, the drug data provisional registration function applies a Laplacian filter to each image to detect the amount of edges in order to remove blurry images such as out-of-focus images from the group of images classified into the same group. Images with less than a predetermined amount can be excluded from the group as unclear images. Images that clearly show unregistered drugs tend to have a large amount of edges detected, while images that are unclear tend to have a small amount of edges. This is because it is possible to improve the accuracy of drug data to be used.

Then, the drug data provisional registration function selects a representative image representing the unregistered drug from among the grouped images of the unregistered drug. Specifically, the drug data provisional registration function is used to obtain the most An image of a nearby unregistered drug can be selected as the representative image. For example, the drug data provisional registration function detects the image of the unregistered drug with the smallest sum of the feature amount differences from other unregistered drugs included in the same group as the image closest to the median value, and It can be selected as a representative image. Here, FIG. 8 is a diagram for explaining a method of selecting a representative image, and shows an example in which each unregistered drug is arranged in a two-dimensional coordinate space according to its feature amount. For example, as shown in FIG. 8A, the feature amount of drug X1 is significantly different from the other unregistered drugs X2 to X7. the sum becomes large. On the other hand, as shown in FIG. 8B, the drug X5 is similar in feature amount to the other unregistered drugs X1 to X4, X6, and X7. , X6 and X7 is the smallest among the drugs X1 to X7 classified in the same group. Therefore, in the example shown in FIG. 8, the drug data provisional registration function can select the image of the unregistered drug X5 as the representative image. The drug data provisional registration function can also obtain the center of gravity corresponding to the feature amount of each unregistered drug, and select the image of the unregistered drug having the feature amount closest to the center of gravity as the representative image. In this embodiment, the drug data provisional registration function is configured to select representative images for the front and back sides of the unregistered drug, but only the image on one side that is stamped/printed is selected as the representative image. can also be configured.

Further, the drug data provisional registration function performs a process of extracting an image of characters or symbols stamped or printed on the unregistered drug from the representative image. Specifically, the drug data provisional registration function uses a machine learning model M3 prepared in advance to detect areas where characters and symbols exist, and detects areas where characters and symbols exist from selected representative images. do. For example, in the example shown in FIG. 9, the drug data provisional registration function uses the machine learning model M3 to obtain the characters "X", "X", "X", "1", "2", and "3". can be detected respectively. Then, the drug data provisional registration function uses the character/symbol classification machine learning model M4 to calculate the feature amount of each detected character or symbol. Furthermore, the drug data provisional registration function identifies each character using the machine learning model M5 for character symbol identification.

Then, the drug data provisional registration function registers drug data (drug provisional master information) of unregistered drugs in the provisional master table 38 . Here, FIG. 10 is a diagram showing an example of the temporary master table 38. As shown in FIG. As shown in FIG. 10, the temporary master table 38 has attributes such as drug identification number (indicated by "No" in FIG. 10), YJ code, drug name, surface feature amount, back side feature amount, area, and shape. Among these attributes, the drug data provisional registration function can automatically register drug identification number, surface feature amount, back side feature amount, area, shape (classification such as circular, elliptical, capsule-shaped, etc.). can. For example, the drug data temporary registration function can register the surface feature amount, back side feature amount, area, shape, etc. of the unregistered drug in the representative image in the temporary master table 38 . Alternatively, the drug data provisional registration function is a function of the surface feature amount, the back side feature amount, and the area of one or more images of unregistered drugs whose coordinates are within a predetermined threshold around the coordinates of the representative image in the coordinate space. Statistical values such as the average value, median value, and mode value can also be configured to be registered in the temporary master table 38 . Among the attributes of the temporary master table 38, the YJ code and drug name can be registered later by the user using an input device.

The drug data provisional registration function also registers character/symbol information of unregistered drugs in the provisional master sub-table 39 for each character or symbol stamped or printed on the unregistered drug. Here, FIG. 11 is a diagram showing an example of the temporary master sub-table 39. As shown in FIG. As shown in FIG. 11, the temporary master sub-table 39 includes drug identification numbers (indicated by "No" in FIG. 11), character symbol identification numbers (indicated by "Sub. No" in FIG. 11), character symbol feature amounts , start point X, start point Y, width, and height, and the drug data provisional registration function can automatically register all of these attributes for each character or symbol of an unregistered drug. For example, in the example shown in FIG. 9, the character symbol information for each of the six characters "X", "X", "X", "1", "2", and "3" is registered in the temporary master sub-table 39. The Rukoto.

In the temporary master sub-table 39, the drug identification number is associated with the drug identification number in the temporary master table 38, making it possible to specify which drug the character/symbol information is. For example, in the example shown in FIG. 9, six characters are stamped or printed on the same drug, so the same drug identification number is registered. In addition, the letter/symbol identification number is a number assigned to each letter or symbol stamped or printed on one drug, and a unique number is assigned according to the number of letter symbols stamped or printed on one drug. . For example, in the example shown in FIG. 9, numbers 1 to 6 are uniquely assigned to six characters "X", "X", "X", "1", "2", and "3". In addition, the starting point X and the starting point Y indicate the position where each character symbol is stamped or printed on each drug, and the starting point of each character symbol in the X direction (for example, the vertical direction in FIG. 9) and the Y direction (for example, 9), which makes it possible to grasp which character symbol is stamped or printed at which position on the medicine. Furthermore, the width and height indicate the size of each character symbol. For example, the width indicates the length in the horizontal direction shown in FIG. 9, and the height indicates the length in the vertical direction shown in FIG.

When the drug data of an unregistered drug is stored in the temporary master table 38 and the temporary master sub-table 39 by the drug data temporary registration function, the inspection function, based on the temporary master table 38 and the temporary master sub-table 39, The medicine imaged in the captured image of the sachet 21 is inspected. That is, in this embodiment, the inspection function inspects the medicine based on the master table 37, and if the medicine enclosed in the sachet 21 is not registered in the master table 37, the temporary master table 38 and Drugs not registered in the master table 37 are inspected based on the temporary master sub-table 39 . Furthermore, when the drug data is not registered in the temporary master table 38, the drug data of the unregistered drug is registered in the temporary master table 38 and the temporary master sub-table 39 by the drug data temporary registration function. Become. Conversely, even if drug data is not registered in the master table 37, if drug data is registered in the temporary master table 38, provisional registration of the unregistered drug is not performed by the drug data provisional registration function, and inspection is performed. Inspection using the temporary master table 38 is performed by the function.

When the inspection function performs inspection using the temporary master table 38 and the temporary master sub-table 39 or using the master table 37, the characteristics of each drug contained in the sachet 21 are By calculating the amount and calculating the degree of similarity with the drug data registered in the master table 37 or the temporary master table 38 using the above formula, it is possible to determine whether the drug described in the prescription is contained in the sachet 21. It can be configured to determine whether or not In addition, the feature amount of the character/symbol information of the drug contained in the sachet 21 is obtained, and similarly, the feature amount of the character/symbol information registered in the temporary master table 38 is used to obtain the degree of similarity of the character/symbol information. By calculating , it is possible to determine whether the character symbols printed/engraved on the medicine match or do not match. As described above, in the present embodiment, character symbols are not recognized by OCR or the like, but similarity is calculated to determine whether they match, thereby determining match/mismatch including font elements. Also, it is possible to determine whether or not there is a match for special character/symbol information such as the logo of a pharmaceutical company, so that more accurate inspection can be performed.

The drug data provisional registration function is a function of transmitting drug data of unregistered drugs stored in the temporary master table 38 and the temporary master sub-table 39 to the external server 40 at regular intervals (for example, once a day). have. The external server 40 can collect drug data of unregistered drugs from a plurality of packaging inspection devices 30 and store them in the database of the external server 40 . Then, the external server 40 refers to the accumulated drug data of the unregistered drugs based on the operation of the system administrator who manages the external server 40, and stores the drug data of the unregistered drugs in the respective package inspection devices 30. can be created as master data to be registered in the master table 37 and delivered to each packaging inspection device 30 as drug data to be registered in the master table 37 .

The inspection result display function causes the display 35 to display a screen showing inspection results by the inspection function. Specifically, the inspection result display function creates screen drawing data for displaying a screen showing the inspection result, and transmits it to the display 35 . The display 35 receives the screen drawing data transmitted from the inspection result display function, and displays a screen showing the inspection result on the monitor of the display 35 . The display 35 has a touch panel, and can be configured to be operable by the user by touching the touch panel. Alternatively, the display 35 can be a liquid crystal monitor or the like, and the user operates another input device to input instructions. may be configured.

FIG. 12 is a diagram for explaining a screen displayed by the display 35 in this embodiment. As shown in FIG. 12, a screen 100 displayed by the display 35 includes an overall information display section 110 that displays overall information collectively indicating inspection results of each of the sachet bags 21 of the series of sachets 2, and a series of sachets. It also has an individual information display section 120 that displays detailed information on the inspection result of the individual sachet 21 of the sachet 2 .

The overall information display section 110 displays a sachet graphic 111 simulating a series of sachets 2 so that the user can grasp the inspection results of the series of sachets 2 at a glance. The package bag graphic 111 has a substantially rectangular configuration in which a plurality of rectangular unit scales 112 are continuously arranged. Each unit scale 112 corresponds to each sachet 21 of the series of sachets 2 on a one-to-one basis. Also, an indicator 113 is displayed to indicate the position of the unit scale 112 corresponding to the sachet 21 displayed on the individual information display portion 120 in the sachet graphic 111 . For example, in the example shown in FIG. 12, an indicator 113, which is a rectangle larger than the unit scale 112, is displayed superimposed on the position of the unit scale 112 corresponding to the sachet 21 displayed on the individual information display section 120. be. Note that the indicator 113 is not limited to the example shown in FIG. For example, it can be configured to be displayed. In this way, by displaying the indicator 113 at the position of the unit scale 112 corresponding to the sachet 21 displayed on the individual information display section 120, the sachet 21 displayed on the individual information display section 120 is , the position of the series of sachets 2 can be easily grasped by the user.

Furthermore, the sachet graphic 111 according to the present embodiment displays the unit scale 112 and the indicator 113 in a color corresponding to the inspection result of each sachet 21 . For example, if the inspection results of all drugs contained in the sachets 21 corresponding to the unit scale 112 conform to the prescription, the unit scale 112 corresponding to the sachets 21 indicates "conforming". displayed in a first color (for example, blue or light blue). On the other hand, if at least one of the drugs contained in the sachet 21 corresponding to the unit scale 112 contains a drug that satisfies the second non-conformance condition and is determined to be "non-conforming", the sachet 21 The corresponding unit scale 112 is displayed in a second color (for example, red) indicating that it has been determined as "non-conforming" that satisfies the second non-conforming condition. Furthermore, among the medicines contained in the sachets 21 corresponding to the unit graduations 112, none of the medicines judged to satisfy the second incompatibility condition are included, but they satisfy the first incompatibility condition and are "unsuitable". , the unit scale 112 corresponding to the sachet 21 is displayed in a third color (for example, yellow).

Note that the indicator 113 is similarly colored and displayed. For example, in the example shown in FIG. 12, among the drugs contained in the sachet 21 displayed by the indicator 113, the drug E satisfies the second incompatible condition and is determined to be "incompatible". It is displayed in a second color (for example, red) indicating that it is determined to be “non-conforming” that satisfies the non-conforming condition No. 2. In this way, the unit scale 112 and the indicator 113 are displayed in different colors according to "conformity", "nonconformance" satisfying the first nonconformity condition, and "nonconformity" satisfying the second nonconformity condition. However, just by glancing at the package bag graphic 111, it is possible to easily grasp what inspection result the package 21 at at which position in the series of package bags 2 is.

The individual information display part 120 is an area for displaying detailed information of an individual sachet 21 designated by the user among the series of sachets 2 . In this embodiment, as shown in FIG. 12, the individual information display unit 120 displays an image information display unit 121 that displays a captured image based on captured image data, and displays detailed information on each drug based on prescription data and drug data. and a button display section 123 for causing the individual information display section 120 to display the sachet 21 desired by the user.

An image captured by the pair of cameras 31 and 32 is displayed on the image information display section 121 . In the present embodiment, the user can alternately display the captured image captured by the camera 31 and the captured image captured by the camera 32 by pressing the reverse button 124 . In addition, in the image information display section 121, a label indicating a drug determined as “unsuitable” by the inspection function is displayed superimposed on the image of the drug. For example, in the example shown in FIG. 12, the drug C is determined to be "unsuitable" satisfying the first unsuitable condition, and the drug E is determined to be "unsuitable" satisfying the second unsuitable condition. In this case, a first indicator (for example, a yellow frame) indicating that the first incompatibility condition is satisfied is displayed superimposed on the image of drug C, and a second indication indicating that the second incompatibility condition is satisfied is displayed. 2 (for example, a red border) is superimposed on the drug E image and displayed. Furthermore, in the image information display section 121, the identification number of each drug, which will be described later, is also superimposed and displayed near the image of each drug.

In addition, as shown in FIG. 12, the drug information display section 122 displays the identification number, image, name, quantity, and inspection result of the drug. The drug identification number is a number for displaying to the user the drug in the image information display section 121 that has been determined as the drug in the drug information display section 122. For example, as drug A in the drug information display section 122, image information When the drug A on the display section 121 is specified, the same identification number “1” as the drug A on the drug information display section 122 is displayed for the drug A on the image information display section 121 . Further, the image of the drug is an image obtained from the drug data, and as shown in FIG. 12, in this embodiment, the photographic image of the front side and the photographic image of the back side of the drug are displayed side by side. The quantity of drug is information obtained from prescription data, and the quantity for one dose is displayed. Furthermore, the inspection result of the drug is displayed as the inspection result by the inspection function, and in this embodiment, as shown in FIG. . Specifically, in the present embodiment, when the type and quantity of the drug are compatible with the prescription and the drug is determined to be “compatible”, “○” is displayed as the inspection result, and the drug satisfies the first non-compatible condition ““. If the drug is determined to be “non-conforming”, the inspection result is displayed as “Δ”, and if the drug is determined to be “non-conforming” that satisfies the second non-conformity condition, the inspection result is displayed as “x”.

Furthermore, in this embodiment, as shown in FIG. 12, a button display section 123 is provided for changing the information of the sachet 21 displayed on the individual information display section 120 . The button display portion 123 includes a button for displaying the information of the preceding sachet 21 in the series of sachets 2 and the button for displaying the information of the next sachet 21 in the series of sachets 2. Buttons for displaying information are displayed, and the user presses these buttons to display the information of the previous or next sachet 21 on the individual information display section 120. be able to. Furthermore, in the button display section 123, information on the preceding sachet 21 determined to be “unsuitable” in the series of sachets 2, or information on the sachets 21 determined to be Skip buttons (fast-forward button, fast-rewind button) for skipping and displaying information on the file wrapper 21 are displayed. By pressing the skip button, the user skips the information of the sachet 21 determined as "conforming" and displays the information of the sachet 21 determined as "non-conforming" on the individual information display section 120. It is possible to efficiently grasp the information of the sachet 21 determined to be "unsuitable".

In this embodiment, when the user presses the button of the button display portion 123 to change the information of the sachet 21 displayed on the individual information display portion 120, the sachet graphic 111 on the overall information display portion 110 Also, the indicator 113 is moved and displayed at the position of the unit scale 112 corresponding to the changed sachet 21 .

Further, the individual information display section 120 is provided with a drug addition button 125 in addition to the buttons of the button display section 123 . For example, if the sachets 21 do not have enough drugs described in the prescription, the user may manually add the drugs to the sachets 21 . In such a case, when the user presses the drug addition button 125, information indicating that the user has added the drug in the sachet 21 can be registered. As a result, the display 35 can display the image of the added drug in the image information display section 121, change the drug inspection result in the drug information display section 122 to "conforming", and display the drug. The unit scale 112 or the indicator 113 of the sachet 21 in the sachet graphic 111 can be changed to a color indicating "fit" and displayed. Although not shown, when the amount of medicine enclosed in the sachet 21 is larger than the prescription, a delete button may be displayed to indicate that the user has taken out the medicine manually. can.

Next, the single package inspection process according to this embodiment will be described. FIG. 13 is a flow chart showing the package inspection process according to this embodiment. The package inspection process shown in FIG. 13 is executed by the package inspection apparatus 30, for example, when the user presses an unillustrated start button for package inspection.

As shown in FIG. 13, in step S101, the prescription data acquisition function of the inspection processing unit 33 performs processing for acquiring prescription data from the receipt computer 10 or the packing machine 20. FIG. Further, in step S102, the image data acquisition function of the inspection processing unit 33 acquires the captured image data captured by the pair of cameras 31 and 32 . Furthermore, in step S103, the master table 37 stored in the storage unit 34 is referred to by the inspection function of the inspection processing unit 33, and the drug data of the drug described in the prescription is acquired from the master table 37. In this embodiment, the inspection function acquires drug data of each drug described in the prescription from the master table 37 using the YJ code of each drug in the prescription data.

Then, in step S104, it is determined whether or not drug data of all the drugs described in the prescription have been acquired from the master table 37 by the inspection function. If the drug data of all drugs described in the prescription can be acquired from the master table 37, the process proceeds to step S108. On the other hand, if the prescription includes a drug for which drug data cannot be obtained from the master table 37, the process proceeds to step S105.

In step S105, the temporary master table 38 and the temporary master sub-table 39 are referred to by the inspection function, and the drug data of the medicine whose drug data is not registered in the master table 37 is stored in the temporary master table 38 and the temporary master sub-table 39. are searched for. Then, in step S106, by referring to the temporary master table 38 and the temporary master sub-table 39 by the inspection function, it is determined whether drug data of all the drugs described in the prescription have been acquired. If the drug data of all the drugs described in the prescription can be acquired from the master table 37, the temporary master table 38 and the temporary master sub-table 39, the process proceeds to step S108. On the other hand, if the prescription includes a drug for which drug data cannot be acquired even after referring to the master table 37, the temporary master table 38, and the temporary master sub-table 39, the process proceeds to step S107.

In step S107, since the drug data is not registered in the master table 37, the temporary master table 38, or the temporary master sub-table 39, a drug data temporary register for temporarily registering the drug data of the unregistered drug whose drug data is not registered is entered. Registration processing is performed. Here, FIG. 14 is a flow chart showing the drug data temporary registration process in step S107.

As shown in FIG. 14, first, in step S201, a process of cutting out an image of each drug from images captured by the cameras 31 and 32 is performed by the drug data temporary registration function. Specifically, the drug data provisional registration function uses the machine learning model M1 for detecting the area where the drug exists, and extracts the sachets from the captured images of the sachet 21 acquired in step S102. The position (area) of each drug included in 21 is detected, and the image of each drug is cut out for each drug.

In step S202, the drug data temporary registration function calculates the feature amount of the drug extracted from the image in step S201 using the machine learning model M2 for classifying the drug image. Then, in step S203, the drug data provisional registration function performs a process of excluding drug images already registered in the master table 37 and the provisional master table 38 from the drug images cut out from the captured image. Specifically, the drug data provisional registration function first performs the drug data registration of the drug registered in the master table 37 acquired in step S103 and the drug data of the drug registered in the temporary master table 38 acquired in step S105. From the data, the characteristic amount of the registered drug, which is the drug described in the prescription and registered in the master table 37 or the temporary master table 38, is obtained. Then, as shown in FIG. 7, the drug data provisional registration function is an inner product of a vector corresponding to the feature amount of each drug extracted from the captured image and a vector corresponding to the feature amount of the registered drug described in the prescription. is calculated as the degree of similarity, and out of the drugs extracted from the captured image, those drugs whose degree of similarity to the drugs registered in the master table 37 or the temporary master table 38 is equal to or greater than a predetermined reference value s are displayed in the master table 37 and As a drug registered in the temporary master table 38, it is excluded from the target of temporary registration of drug data.

In step S204, the drug data provisional registration function groups images of unregistered drugs that have not been excluded from provisional registration targets in step S203. Specifically, in the drug data provisional registration function, similarly to step S203, the inner product of the vectors corresponding to the characteristic amounts of the two unregistered drugs to be compared is equal to or greater than a predetermined reference value s (0<s<1). , if two unregistered drugs are similar, the two unregistered drugs are classified into the same group, and the inner product of the vectors corresponding to the feature amounts of the two unregistered drugs is less than a predetermined reference value s; If two unregistered drugs are dissimilar, classify the two unregistered drugs into another group.

In step S205, a process of selecting a representative image from among the images of a plurality of unregistered drugs classified into the same group in step S204 is performed by the drug data temporary registration function. Specifically, the drug data provisional registration function uses the image of the unregistered drug in which the feature amount of the unregistered drug calculated in step S202 is closest to the median value, mode value, or average value in the group as a representative image. can be selected as For example, in the present embodiment, the drug data provisional registration function detects the image of the unregistered drug with the smallest sum of feature amount differences from other unregistered drugs belonging to the same group as the image closest to the median value. , can be selected as a representative image.

In step S206, processing for cutting out a character/symbol area from the representative image selected in step S205 is performed by the drug data temporary registration function. Specifically, the drug data provisional registration function uses a machine learning model M3 for detecting character/symbol areas that has been prepared in advance to extract areas containing characters and symbols from the representative image selected in step S205. Detect by symbol. Furthermore, in step S207, the feature amount of each character symbol acquired in step S206 is calculated using the machine learning model M4 for classifying the character symbol by the drug data temporary registration function. Then, in step S208, the drug data provisional registration function uses the machine learning model M5 for specifying character symbols to identify each character symbol based on the feature amount of each character symbol.

In step S209, the drug data provisional registration function automatically registers a unique drug identification number and character/symbol identification number in the temporary master table 38 and the temporary master sub-table 39, respectively. The feature amount, area, and shape are automatically registered in the temporary master table 38, and the feature amount, start point X, start point Y, width, and height information of the character/symbol area are automatically registered in the temporary master sub-table 39.

Since the YJ code and drug name in the temporary master table 38 cannot be automatically registered, the user is requested to register the YJ code and drug name in the temporary master table 38 by the drug data temporary registration function in step S210. A message is output to . As a result, the user uses an input device (not shown) to register the YJ code and drug name in the temporary master table 38, so that the drug data of the unregistered drug is registered in the temporary master table 38 as shown in FIG. to be registered. The process waits until the user registers the YJ code and drug name in the temporary master table 38 (step S211=No), and when the registration is completed (step S211=Yes), the drug data temporary registration process of FIG. , and proceeds to step S108 shown in FIG.

Returning to FIG. 13, in step S108, inspection processing of the packaged medicine is performed by the inspection function. Specifically, the inspection function includes the drug data obtained from the master table 37 in step S103, the drug data obtained from the temporary master table 38 and the temporary master sub-table 39 in step S105, and/or the drug data provisional data in step S107. Using the drug data provisionally registered in the registration process, an inspection process is performed to check whether the types and quantities of the drugs packaged in the sachet 21 match the prescription. Specifically, the inspection function identifies the drug and the quantity described in the prescription, and determines whether each drug described in the prescription is present in the captured image and whether the quantity matches. More specifically, the inspection function compares an image of each drug in the captured image with an image of drug data corresponding to the drug in the prescription, and compares the drug in the prescription with the size, shape, number, color and stamp. If the required number of medicines matching the printed image exists in the captured image, it is determined that the medicine of the prescription is “suitable” for the prescription.

On the other hand, the inspection function cannot identify the medicine with sufficient accuracy from the captured image in the case of the medicine without engraving/printing among the medicines in the prescription. Judged as "nonconforming" that does not meet the conditions. Furthermore, in the present embodiment, the inspection function is performed when the drug of the prescription is an unregistered drug that is not registered in the master table 37 or the temporary master table 38, or if the first nonconformity condition is not satisfied, but the image is captured. If the required quantity of matching drugs in the image cannot be identified, then the prescription drug is determined to be "non-conforming", which satisfies the second non-conformance condition. Furthermore, the inspection function inspects all the sachets 21 of the series of sachets 2 and stores the inspection results for each sachet 21 in the storage unit 34 .

In step S109, the inspection result displaying the inspection result of step S108 is displayed by the inspection result display function. Specifically, the inspection result display function displays a sachet graphic 111 consisting of a unit scale 112 corresponding to each sachet 21 of a series of sachets 2, and displays the inspection result of each sachet 21. Accordingly, each unit scale 112 is colored a first color (e.g., light blue) to indicate that all medications comply with the prescription, and a second color to indicate that at least one medication satisfies a second non-compliance condition. colored with a third color (eg, yellow) to indicate that it contains no drugs that meet the second incompatibility condition, but at least one drug that satisfies the first incompatibility condition (eg, red). The sachet graphic 111 is drawn as shown.

Moreover, the inspection result display function draws an indicator 113 for indicating the position of the sachet 21 displayed in the individual information display section 120 among the series of sachets 2 in the sachet graphic 111 . Note that the indicator 113 also has a first color (for example, light blue), a A second color (e.g., red) to indicate that it contains at least one drug that satisfies 2 mismatch conditions, does not contain any drugs that satisfy the second mismatch condition, but contains one drug that satisfies the first mismatch condition Indicator 113 is drawn so that it is colored with a third color (eg yellow) to indicate that it contains When the screen 100 is displayed for the first time, the indicator 113 can be set to be displayed at the position of the first sachet 21 in the series of sachets 2 .

Furthermore, the inspection result display function displays the image information so that the captured image of the captured image data of the sachet 21 designated by the user, among the captured image data acquired in step S102, is drawn on the image information display unit 121. Draw the display unit 121 . In addition, the inspection result display function superimposes the identification number on the image of each drug in the captured image that can be identified based on the inspection result of step S108, and displays the image of the drug determined as "nonconforming" is superimposed with a first label indicating that the drug is "incompatible" satisfying the first incompatibility condition, or a second label indicating that the drug is "incompatible" with the second incompatibility condition. to draw.

In this embodiment, the inspection result display function displays the drug image, drug name, and quantity for each drug described in the prescription in the drug information display section 122 based on the drug data described in the prescription. draw. Further, the inspection result display function assigns an identification number to each medicine described in the prescription, and draws so that it can be displayed in association with each medicine. Furthermore, the investigation result display function draws the inspection results of each drug obtained in step S108 so that they can be displayed in association with each drug.

It should be noted that the inspection result display function allows the user to change the information of the sachet 21 displayed on the individual information display unit 120 by pressing a button on the button display unit 123 or the like, so that the changed packaging can be displayed. The drug information is displayed on the display 35.

As described above, the single-package inspection device 30 according to the present embodiment acquires prescription data, acquires drug data from the master table 37, and based on the acquired prescription data and drug data, In addition to the function of performing a normal inspection to determine the suitability of the drug enclosed in the camera, if the drug data of the drug included in the prescription is an unregistered drug not registered in the master table 37, It has a function of creating and registering a temporary master table 38 of unregistered medicines from the captured images taken by 31 and 32, and referring to the temporary master table 38 to determine compatibility of the unregistered medicines. As a result, the package inspection device 30 according to the present embodiment is not registered in the master information received from the external server 40. Therefore, even an unregistered drug whose drug data is not registered in the master table 37, Since the package inspection device 30 can create drug data of unregistered drugs and register them in the temporary master table 38, the temporary master table 38 can be referenced to inspect the unregistered drugs.

In addition, in the present embodiment, when the unregistered drug has a shape that does not require extraction of character/symbol information, the image of the unregistered drug is smoothed and then grouped so as to be printed. Even for medicines such as capsules whose characters and symbols change at the timing of imaging, it is possible to perform inspection using the medicine data registered in the temporary master table 38 . That is, characters are often printed in the circumferential direction on the capsule, and in this case, the character captured in the captured image changes depending on the rolling position of the capsule. Therefore, if the character/symbol information is registered in the temporary master sub-table 39 as it is, the characters of the medicine data registered in the temporary master sub-table 39 do not match the characters captured in the captured image, and the capsules can be correctly detected. It may not be possible to specify. Therefore, by reducing the influence of characters and symbols by smoothing, it is possible to identify the capsule.

On the other hand, when the unregistered drug has a shape that requires extraction of character information, an area in which character information exists is detected from the representative image of the unregistered drug using a machine learning model M3 for character area detection. , each character in the area where the detected character information exists is identified using a machine learning model M4 for character identification. As a result, if the medicine has a shape such as a tablet that allows characters or symbols to be identified, the accuracy of inspection can be further improved by identifying even the characters or symbols stamped or printed on the medicine.

Although preferred embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the description of the above embodiments. Various modifications and improvements can be added to the above-described embodiment examples, and forms with such modifications and improvements are also included in the technical scope of the present invention.

For example, in the above-described embodiment, the scene in which two or more types of drugs are packaged has been exemplified and explained, but the present invention is not limited to this. It can also be configured to determine whether it is enclosed by conforming to.

DESCRIPTION OF SYMBOLS 1... Package inspection system 10... Receipt computer 20... Packaging machine 30... Package inspection apparatus 31, 32... Camera 33... Inspection processing part 34... Storage part 35... Display 36... Transport part 37... Master table 38... Temporary Master table 39 Temporary master sub-table 40 External server 2 Series of package bags 21 Package package 100 Screen 110 Overall information display section 111 Package package graphic 112 Unit scale 113 Indicator 114 Sorting Button 120... Individual information display section 121... Image information display section 122... Drug information display section 123... Button display section 124... Reverse button 125... Add drug button 200... Pharmacy

Claims (9)

A packaging inspection device for inspecting whether a drug packaged in each packaging bag of a series of connected packaging bags conforms to a prescription,
an imaging device for imaging the sachet;
a storage device that stores drug master information;
an inspection processing device for determining suitability of the drug enclosed in the sachet,
The inspection processing device
acquisition means for acquiring prescription data and drug master information;
normal inspection means for determining suitability of the drug based on the prescription data and the drug master information;
When drug data of a drug included in the prescription is an unregistered drug that is not registered as the drug master information, drug temporary master information of the unregistered drug is created from an image captured by the imaging device and registered. a drug data temporary registration means;
provisional master inspection means for determining suitability of the unregistered drug based on the provisional drug master information;
The drug data temporary registration means calculates a feature amount of the unregistered drug from the captured image using a machine learning model, and classifies the unregistered drug into groups based on the calculated feature amount of the unregistered drug. grouping, and creates the drug temporary master information for each of the unregistered drugs classified into the same group . When the inner product of the vector corresponding to the feature quantity of one of the unregistered drugs and the vector corresponding to the feature quantity of the other unregistered drug is equal to or less than a predetermined reference value, the drug data provisional registration means 2. The package inspection device according to claim 1 , wherein one unregistered drug and said other unregistered drug are classified into the same group. The drug data temporary registration means selects a representative image from among the images of the plurality of unregistered drugs classified into the same group, based on the feature amounts of the plurality of unregistered drugs classified into the same group. 3. The unpackaged inspection apparatus according to claim 1 , wherein the drug provisional master information is created based on the selected representative image. The drug data provisional registration means obtains a median value of feature amounts of a plurality of unregistered drugs classified into the same group, and selects the median value from images of the plurality of unregistered drugs classified into the same group. 4. The single package inspection device according to claim 3 , wherein the image of the unregistered drug having the closest feature amount is selected as the representative image. The drug data temporary registration means has a function of determining the shape of the unregistered drug using a machine learning model for shape determination,
3. The package according to claim 1 or 2 , wherein when the unregistered drug has a shape that does not require extraction of character information, the image of the unregistered drug is smoothed before the grouping. Chemical inspection device. The drug data temporary registration means
When the unregistered drug has a shape that requires extraction of character information, areas where each character information exists are identified from the image of the unregistered drug in the captured image , and a machine learning model for character area detection is used. a function to detect using
5. The package inspection apparatus according to any one of claims 1 to 4 , further comprising a function of specifying each character in an area where detected character information exists using a machine learning model for character specification. The drug data temporary registration means performs edge amount calculation processing on the image of the unregistered drug, and excludes the unregistered drug whose calculated edge amount exceeds a predetermined value from the group of unregistered drugs. Item 7. The package inspection device according to any one of items 1 to 6 . A packaging inspection program for causing a computer to inspect whether a drug packaged in each packaging bag of a series of connected packaging bags conforms to a prescription,
the computer,
acquisition means for acquiring prescription data and drug master information;
normal inspection means for determining suitability of the drug based on prescription data and drug master information;
Based on the prescription data and the drug master information, when the drug data of the drug included in the prescription is an unregistered drug that is not registered in the drug master information, the unregistered drug is detected from the captured image captured by the imaging device. drug data temporary registration means for creating and registering drug temporary master information of registered drugs;
Functioning as a temporary master inspection means for judging compatibility of the unregistered medicine based on the temporary medicine master information ,
The drug data temporary registration means calculates a feature amount of the unregistered drug using a machine learning model from the captured image captured by the imaging device, and based on the calculated feature amount of the unregistered drug, A package inspection program for performing grouping for classifying drugs into groups and creating the drug temporary master information for each of the unregistered drugs classified into the same group. A package inspection method for inspecting whether a drug packaged in each package bag of a series of connected package bags conforms to a prescription, comprising:
an obtaining step of obtaining prescription data and drug master information;
a normal inspection step of determining suitability of the drug based on prescription data and drug master information;
When the medicine enclosed in the sachet is an unregistered medicine that is not registered as the medicine master information, temporary medicine master information of the unregistered medicine is created from the captured image of the unregistered medicine and registered. a drug data provisional registration step;
performing a provisional master inspection step of determining suitability of the unregistered drug based on the provisional drug master information;
In the drug data provisional registration step, a feature amount of the unregistered drug is calculated from the captured image using a machine learning model, and the unregistered drug is classified into groups based on the calculated feature amount of the unregistered drug. grouping, and creating the drug temporary master information for each of the unregistered drugs classified into the same group .