JP2023005927A - Medicine discrimination system and medicine discrimination method - Google Patents

Abstract

To properly carry out calculation of the number of medicine and discrimination of a group of medicine in medicine discrimination based on a captured image.SOLUTION: An imaging unit 20 has a range-finding sensor which can detect a distance between the imaging unit 20 and an imaging object. The imaging unit 20 transmits a captured image to a medicine discrimination unit 30 together with distance information to the imaging object. The medicine discrimination unit 30 calculates the number of the medicine based on the distance information with respect to the captured image transmitted by the imaging unit 20, and discriminates, as a group of the medicine, the imaging object surrounded by a region at a position within a distance of a package discrimination range.SELECTED DRAWING: Figure 1

Description

TECHNICAL FIELD The present invention relates to a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug. The present invention also relates to a drug identification method for identifying a photographed drug by performing image recognition of a photographed image of the drug.

Mechanical identification of drugs is often required in drug handling facilities such as pharmacies. For example, mechanical identification of drugs is used to determine whether or not drugs arranged based on a prescription actually match the contents of the prescription by an entity other than the operator (pharmacist, etc.). A device (medicine inspection device) may be used. By using such a device, it is possible to prevent operational errors in delivering drugs to patients.

Patent Literature 1 describes a tablet identification device that photographs tablets or the like on a conveyer and recognizes the printed content on the surface of the tablet or the like from the photographed image. With this tablet identification device, the system can automatically recognize the type of tablet or the like being conveyed from the printed content on the surface while the tablet or capsule is being conveyed on the conveyer.

JP 2015-054115 A

In order for the identification device for tablets, etc. disclosed in Patent Literature 1 to perform recognition correctly, it is necessary that the tablets, etc. to be recognized are conveyed as single units (unwrapped tablets) that are not wrapped. However, many ready-made drugs are delivered to pharmacies and the like in a packaged state.

In particular, in the case of tablets, many of them are distributed in a state where about 10 tablets are enclosed in one PTP sheet. In addition, for powders and granules (powders), there are cases where a unit (one package) of SP packaging containing powder or granules is sold as a single sheet by connecting multiple agents. many.

In the case where a plurality of medicines are enclosed in a single package as described above, so to speak, as a group of medicines, image recognition by the conventional method can correct the number of medicines and the range of the medicines in the group. It may not be recognized.

That is, individual drugs (of the same type) included in a group of drugs may be recognized as separate types of drugs, or the entire group of drugs may be recognized as one drug (the total number of drugs is one). sometimes.

Therefore, in drug identification based on a photographed image, the present invention correctly identifies the quantity of drugs and appropriately cuts the group of drugs even when a plurality of drugs that form a group are enclosed in a package. for the purpose.

In order to solve the above problems, a drug identification system as an example of an embodiment according to the present invention is a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug. and a drug identification unit for identifying the drug contained in the captured image, the imaging unit includes a distance sensor capable of detecting the distance from the imaging unit to the imaging target, The photographed image is transmitted to the drug identification unit together with distance information to the object to be photographed, and the drug identification unit calculates the number of drugs contained in the photographed image transmitted from the photographing unit based on the distance information. characterized by

Further, preferably, the drug identification unit identifies, as a first thickness region, a region having a thickness greater than or equal to a reference thickness detectable by the distance measuring sensor in the object to be imaged, based on the distance information. is identified as a second thickness region, an imaging target surrounded by the first thickness region is identified as a group of drugs, and the first 2 is calculated as the number of medicines.

Further, preferably, the drug identification unit identifies, in the captured image transmitted from the imaging unit, an imaging target surrounded by an area located at a distance within the first identification range as a group of drugs.

Further, preferably, the drug identification unit counts the number of regions located within the second identification range for the imaging target identified as the group of drugs, and determines whether the drug is included in the group of drugs. It is preferable to calculate the number of drugs to be used.

Preferably, the distance measuring sensor uses LiDAR (Light Detection and Ranging).

Further, a drug identification method as another example of an embodiment according to the present invention is a drug identification method for identifying a photographed drug by performing image recognition of a photographed image of the drug. and a drug identification step for identifying drugs included in the photographed image, wherein the photographing step detects the thickness of an object to be photographed, and the drug identification step detects a thickness equal to or greater than a predetermined packaging identification dimension. By identifying an imaging target surrounded by areas having It is characterized by calculating the number of drugs contained in the drug.

According to the medicine identification system and the medicine identification method as an example of the embodiment of the present invention, the number of medicines contained in the photographed image and the number of medicines in a group are determined based on the distance between the imaging unit and the imaging target or the thickness of the imaging target. A range of drugs can be correctly identified.

In addition, by identifying a region within the group of drugs within the range of quantity identification, it is possible to correctly identify how many individual drugs are included in the group of drugs.

1 is a block diagram schematically showing the configuration of a drug identification system as an example of an embodiment according to the present invention; FIG. FIG. 2 is a cross-sectional view showing the thickness of each portion of the drug package in the drug identification system of FIG. 1; 2 is a flowchart showing a drug identification procedure in the drug identification system of FIG. 1; 4 is a flowchart showing the procedure of edge detection; 4 is a flow chart showing a procedure for calculating the number of pieces;

The block diagram of FIG. 1 schematically shows the configuration of a drug identification system 10 as an example of an embodiment according to the present invention. As an example of a drug 12 to be identified in this drug identification system 10, FIG. 1 shows a state in which a tablet 14 is enclosed in a PTP package 13 containing 10 tablets.

A drug identification system 10 shown in FIG. An identification unit 30 and a medicine identification unit 50 for identifying the type of medicine 12 are provided. These components may be integrated into one device, or may be divided and arranged in a plurality of devices.

For example, the drug identification system 10 may be constructed in one smartphone with the screen of the smartphone as the monitor 60, the camera incorporated in the smartphone as the imaging unit 20, and the memory and processor as other components. Alternatively, the drug identification system 10 is built on a network (such as the Internet), and each of a plurality of devices connected via the network functions as a component, The configuration may be such that the drug identification system 10 can be used from devices at various locations.

In FIG. 1, drug 12 is placed on base 11 . If the drug identification system 10 is a dedicated device provided in a pharmacy, the drug placement pedestal (for example, a black plate) provided in the dedicated device becomes the pedestal 11 . In the case where the drug identification system 10 is constructed by a smartphone, the pedestal 11 may be any object such as a work desk as long as the drug 12 can be placed stably. It is preferable that the pedestal 11 has a flat surface.

(basic operation)
A user (operator) who intends to perform identification work using this medicine identification system 10 first captures the medicine 12 (the area containing the medicine 12) with the imaging unit 20 to obtain a photographed image. Then, this photographed image is input to the medicine identifying section 30 . The drug identification unit 30 identifies which part of a drug contained (pictured) in a photographed image is a group of drugs, how many drugs are included in the group of drugs, and the like. . After that, the drug identification unit 30 performs predetermined processing (such as conversion into a data format accepted by the drug identification unit 50 ) for inputting the captured image to the drug identification unit 50 , and sends the captured image to the drug identification unit 50 . is passed as input data.

The drug identification unit 50 identifies the type of drug included in the captured image and returns the identification result to the drug identification unit 30 . After that, the drug identification unit 30 transmits information about the identified quantity of drugs and the specified type of drug to the monitor 60 . The monitor 60 displays the information transmitted from the drug identification unit 30 to the user. That is, information such as the number and type of medicines 12 captured by the imaging unit 20 is displayed on the monitor 60 . As a result, the user of the drug identification system 10 can know the identification results such as the number and type of drugs 12 to be imaged.

(About range sensor)
In this embodiment, the photographing unit 20 incorporates a distance measuring sensor. A ranging sensor is a sensor capable of detecting the distance from the sensor to the object to be photographed. For example, this distance measuring sensor can be constructed by combining an element that projects infrared rays and a sensor that detects infrared rays reflected from an object to be photographed. When this combination is used, the distance to the photographing object can be calculated from the time elapsed from the projection of the infrared rays to the detection of the reflected infrared rays (calculated from the phase of the infrared rays, the pulse shift, etc.).

As the ranging sensor, one using LiDAR (Light Detection and Ranging) may be used. In other words, the distance, position, shape, etc. of the object to be photographed can be measured by irradiating the object to be photographed with a laser beam while scanning and measuring the time it takes for the laser beam to return after being reflected by the object to be photographed. may be used. Since some smartphones in recent years (especially after 2020) are equipped with such LiDAR, it is also possible for the smartphone to function as the imaging unit 20 equipped with LiDAR.

Note that any distance measuring sensor may be used as long as it can detect the distance to the object. There are various distance detection methods other than the ToF (Time of Flight) method, which measures the distance based on the time elapsed from the projection of light to the detection of its reflection. For example, methods such as a stereo vision method and a structured light method are known. The stereo vision method is a distance detection method using a so-called 3D camera. Typically, the depth distance is calculated by triangulation from the parallax between images captured by two or more cameras. The structured light method is a method that uses a projector that projects a reference pattern (pattern) onto an object and a camera that captures the projected reference pattern. Typically, a camera confirms how the reference pattern projected onto the object is distorted, and the depth distance is calculated from the degree of distortion. Each method has its own advantages, but the ToF method such as LiDAR has the advantage of being able to detect distance regardless of lighting conditions (for example, even in dark places).

(Edge detection using distance)
By using the distance measuring sensor, it becomes possible to detect the edge of the object to be photographed, here the contour line of the medicine package for packaging the medicine 12 . Since the shape of the drug package is standardized, its thickness falls within the specified range. Therefore, by calculating the thickness of the object to be photographed based on the distance from the sensor to the object to be photographed detected by the distance measuring sensor, the area having the thickness within a predetermined range is the area where the drug package exists. can be specified.

The drug package generally has a pocket portion containing the drug 12 and a peripheral edge (not containing the drug 12) surrounding the pocket portion. In particular, the thickness of the peripheral portion falls within the range specified (standardized) as described above. The distance from the range-finding sensor differs between the peripheral portion where the medicine 12 is not stored and the outside of the peripheral portion (for example, the pedestal on which the medicine 12 is arranged). Therefore, the drug identification unit 30 that receives the data of the photographed image identifies which range in the photographed image is the peripheral portion (edge) based on the detection result of the distance measuring sensor (distance information to the photographed object). It is possible to

If the pedestal 11 is flat and the imaging range is limited within the pedestal 11, the pedestal 11 is detected as being the farthest point from the distance measuring sensor in the imaging target. Using the point of the pedestal 11 as a reference position, the drug identifying unit 30 can calculate the "thickness" of the imaging target by calculating how close the imaging target is to the reference position.

For example, as shown in FIG. 2, when the drug 12 is placed on a flat pedestal 11, the distance from the distance measuring sensor (imaging unit 20) to the pedestal 11 is D, and the thickness of the periphery of the drug 12 is D. If d1, the distance from the range sensor to the periphery is detected as (D-d1). Based on this, the drug identification unit 30 calculates the difference between the distance D of the reference position and the distance (D−d1) of the peripheral portion, that is, (D−(D−d1))=d1 as the thickness of the peripheral portion. can be done.

For example, in the PTP packaging 13, the thickness d1 of the peripheral portion is generally about 200 μm to 300 μm. Therefore, when the numerical value range from 200 μm to 300 μm is set as the package identification range (first identification range), the drug identification unit 30 detects an area having a thickness within the package identification range (distance within the first identification range) in the imaging target ) can be identified as one PTP package 13, that is, a group of drugs.

More specifically, a group of medicines is a portion in which regions (peripheral portions) having a thickness within the package identification range are connected. It can be said that the pocket portion is also surrounded by the peripheral portion, but it is not the individual pockets that are identified as the “group of drugs”, but the peripheral portion and all of the pockets contained within. This is a summary range. That is, the area surrounded by the contour lines (edges) of the peripheral portion is a group of medicines. Since the thickness of the pocket portion is generally greater than that of the peripheral edge portion, a "continuous region having a thickness equal to or greater than the package identification range" may be identified as a "group of medicines."

In addition, when there are two or more portions surrounded by regions having a thickness within the package identification range in the photographed image, two or more groups of drugs (two groups or more) are present. When two or more types of drugs are placed on the base 11 at the same time, the number of "group of drugs" increases according to the number of types of drugs. Therefore, the drug identification unit 30 can check how many types of drugs are to be identified by identifying how many "group of drugs" are included in the captured image.

(Number calculation using distance)
Furthermore, in the case of capsules and tablets, the thickness of the pocket for containing the drug 12 itself is also standardized. The thickness of the pocket portion for storing the drug 12 in the case of powders and granules varies somewhat depending on the degree of aggregation of the powders and granules, but the thickness falls within a certain range according to the packing amount of one package. Therefore, based on the detected thickness, it is possible to identify the area containing the drug 12 in the captured image. Accordingly, by totaling the number of regions in which the medicines 12 are stored in the photographed image, it is possible to calculate the number of medicines 12 (the number of tablets, the number of packets) in the photographed image.

(Group range detection using distance)
For example, it is assumed that the thickness of the pocket portion of the PTP packaging 13 for the medicine 12 in FIG. 2 is defined as 4500 μm to 5000 μm. In this case, the drug identification unit 30 sets a numerical value range of 4500 μm to 5000 μm as a numerical value identification range (second numerical range), and an area (a distance within the second identification range) having a thickness within the numerical identification range in the imaging target. The area where the tablets 14 are located can be identified as pockets in which the tablets 14 are individually placed. Therefore, the drug identification unit 30 can calculate the number of tablets 14 in FIG. can.

Here, when there are a plurality of "groups of drugs" described above (a plurality of drugs are objects of identification), the above number calculation is preferably performed for each "group of drugs". Thereby, the medicine identification unit 30 can calculate how many medicines of each kind exist for each kind of medicine.

(Drug Identification by Medicine Identification Unit 50)
As described above, the drug identification system 10 of the present embodiment can identify the number of drug types and the number of each type of drug. The type of drug (what kind of drug it is) is specified by the drug specifying unit 50 .

The drug identification unit 50 identifies the type of the captured drug based on the captured image. In specifying this drug, for example, it is possible to specify using a specific indicator written on the drug package. The specific index is written in the form of engraving, printing, or the like on the drug 12 or the drug package that packages the drug 12, and indicates what the drug 12 is, that is, specific information on the drug 12. Specifically, optical marks such as bar codes and two-dimensional codes can be used as specific indicators. The drug identification unit 50 acquires a specific index made up of characters or barcodes from the photographed image by image analysis, and compares the specific information indicated by the obtained specific index with a specific index table prepared in advance, thereby identifying the drug 12. You can get information about the type. When the information on the drug 12 is obtained based on the specific information as described above, the drug identification unit 50 transmits the information on the drug 12 obtained based on the specific information to the drug identification unit 30 as the identification result. .

Note that when the drug identification unit 30 identifies that a plurality of types of drugs are included in the captured image, the drug type identification by the drug identification unit 50 is preferably performed for each type of drug. . That is, the drug identifying unit 50 identifies the type of the group of drugs based on the specific index included in the “group of drugs” identified by the drug identifying unit 30 . When there are a plurality of groups of medicines in a group, this specifying operation is performed for each group of medicines.

As described above, when the identification of the number of types of drugs included in the captured image, the number of each type of drug, and the type of each drug are completed by the drug identification unit 30 and the drug identification unit 50, the result is information is sent to the monitor 60 .

A monitor 60 displays the transmitted information to the user. The information displayed on the monitor 60 indicates, for each group of drugs identified by the drug identification unit 30, the number of drugs included in the group and the types of drugs in the group. As described above, according to the drug identification system 10 of the present embodiment, when a group of a plurality of drugs (tablets 14 in FIG. 1) are enclosed in a package (PTP package in FIG. 1), the Medications enclosed in a bundled package can be collectively identified as a group of medications. Therefore, it is possible to correctly identify the number of types of drugs to be imaged and the number of each type of drug.

(Procedure for identification by drug identification system 10)
FIG. 3 is a flow chart showing the procedure for identifying the medicine 12 (procedure of the medicine identification method) performed by the medicine identification system 10 of FIG. First, the drug 12 is photographed by the photographing unit 20 having a distance measuring sensor to obtain a photographed image (step S01). This photographed image is input to the drug identification unit 30, and the drug identification unit 30 starts image recognition of the input photographed image (step S02), and confirms the result of distance detection by the distance measuring sensor (step S03).

Edge detection of the medicine package is performed based on the result of the distance detection (step S04). Details of edge detection will be described later, but based on the result of edge detection, the number of "group of drugs" (the number of types of drugs) in the captured image is identified (step S05).

Further, the number of drugs in each group is calculated for each drug in the group (step S06). Although the content of the number calculation will be described later, the number of drugs contained in each group is calculated as a result of the number calculation.

Then, the drug identification unit 30 inputs the photographed image to the drug identification unit 50, and the drug identification unit 50 identifies the type of drug based on the specific index included in the photographed image (step S07). The drug identification result is sent to the drug identification unit 30, and the drug identification unit 30 monitors 60 the information on the number of types of identified drugs, the number of each type of drug, and information on the type of each identified drug. (step S08). The monitor 60 displays the information (identification result) transmitted from the drug identification unit 30 to the user (step S09).

(About edge detection)
The edge detection procedure performed in step S04 described above is shown in the flow chart of FIG. The drug identification unit 30 determines the distance from the imaging unit 20 to the base 11 as a reference distance based on the distance information of the imaging target detected by the distance measuring sensor (step A1). This reference distance is the distance between two planes when the photographing unit 20 and the pedestal 11 are arranged in two planes parallel to each other, that is, the vertical distance from the photographing unit 20 to the pedestal 11. . When the distance measuring sensor is a LiDAR, the vertical distance to the pedestal 11 can be calculated from the farthest position in the captured image from the imaging unit 20 and the irradiation angle of the laser beam irradiated toward that position. It is possible. Further, when the optical characteristics of the base 11 (for example, the base 11 is colored in solid black, the specific pattern is drawn, etc.) are known in advance, the optical characteristics of the base 11 can be identified in the photographed image. The vertical distance between the matching location and the photographing unit 20 can be used as the reference distance.

Subsequently, the drug identification unit 30 detects how close each area in the captured image is to the imaging unit 20 from the reference distance, that is, the thickness of each area (step A2). Then, it is checked at which position in the photographed image there is an area having a thickness within a predetermined package identification range (step A3). The predetermined package identification range is, in other words, a numerical range of the thickness of the peripheral portion of the drug package assumed to be photographed.

Then, the drug identification unit 30 identifies the area surrounded by the edge of the drug package as the area having the thickness within the package identification range in the photographed image as a group of drugs (step A4). Here, when there are a plurality of portions surrounded by edges in the photographed image, each portion is identified as a group of drugs that are separate from each other. As described above, the drug identification unit 30 identifies a group of drugs included in the captured image.

(Regarding number calculation)
The flow chart of FIG. 5 shows the number calculation procedure performed in step S06. The drug identification unit 30 counts the number of regions having a thickness within a predetermined quantity identification range, with respect to the thickness of the regions in the captured image detected by the edge detection described above (step B1). The predetermined quantity identification range is, in other words, a numerical range of the thickness of the pocket portion of the drug package assumed to be photographed, for example, a range of 4500 μm to 5000 μm. Further, when there are a plurality of groups of medicines in one group, the number of medicines contained in each group is calculated for each group (step B2). As described above, the medicine identification unit 30 calculates the number of medicines contained in each of a group of medicines.

The thickness of the pocket portion differs between PTP packaging used for tablets and capsules and SP packaging used for powders and granules. Also in PTP packaging, the thickness of the pocket may vary depending on the size of the drug to be packaged. Since a plurality of thicknesses are assumed for the pocket portion in this manner, a plurality of types of quantity identification ranges may be determined in advance, and an area matching one of the quantity identification ranges may be counted as a pocket portion. . Alternatively, any location having a thickness greater than the aforementioned package identifier range may be counted as a pocket. In this case, all values larger than the package identification range are the numerical range of the quantity identification range.

Further, in the above embodiment, the thickness (distance from the photographing unit 20) of each of the peripheral portion and the pocket portion is quantitatively detected as a numerical value, but an accurate numerical value is necessarily detected. It is not necessary, and it is sufficient if it is possible to distinguish between the peripheral portion and the pocket portion. For example, the distance information provided by the distance measuring sensor to the drug identification unit 30 is not a numerical value of thickness, but a “thin region” (first thickness region) having a certain thickness or more in the imaging target and a “thin region”. It may be information indicating that there is a “thick region” (second thickness region) having a large thickness.

Specifically, the area of the object to be imaged that has a thickness greater than or equal to the reference thickness that can be detected by the distance measuring sensor (the area that is at least slightly closer to the imaging unit 20 than the pedestal 11) is the first thickness regardless of the thickness. region”, and a region having a greater thickness than the first thickness region (a region closer to the imaging unit 20 in the “first thickness region”) is identified as a “second thickness region”. I hope it is. In this case, a continuous portion surrounded by the "first thickness region" (that is, peripheral portion) of the imaging target is identified as a "group of drugs". Then, the number of "second thickness regions" (that is, pocket portions) included inside the "group of drugs" is calculated as the number of drugs.

As described above, according to the drug identification method of the present embodiment, when the identification result of the drug 12 targeted by the user to be imaged is displayed on the monitor 60, not only the type of the drug 12 but also the number of the drug 12 are displayed. display is possible. Furthermore, even if a plurality of similar drug packages are included, they are correctly identified as separate drugs 12 .

REFERENCE SIGNS LIST 10 drug identification system 12 drug 20 imaging unit 30 drug identification unit 50 drug identification unit 60 monitor

Claims (6)

In a drug identification system that identifies a photographed drug by performing image recognition of a photographed image of the drug,
an imaging unit for imaging the drug;
a drug identification unit that identifies a drug included in a captured image;
with
the photographing unit includes a distance sensor capable of detecting a distance from the photographing unit to an object to be photographed, and transmits the photographed image together with distance information to the object to be photographed to the drug identification unit;
The drug identification system, wherein the drug identification unit calculates the number of drugs included in the captured image transmitted from the imaging unit based on the distance information. Based on the distance information, the drug identification unit identifies, as a first thickness region, a region having a thickness greater than or equal to a reference thickness detectable by the distance measuring sensor in the object to be imaged. identifies the thicker region as the second thicker region,
Identifying an imaging target surrounded by the first thickness region as a group of drugs,
2. The medicine identification system according to claim 1, wherein the number of said second thickness regions included inside said group of medicines is calculated as the number of medicines. The drug identification unit, for the captured image transmitted from the imaging unit,
2. The medicine identification system according to claim 1, wherein an object to be imaged surrounded by an area located at a distance within the first identification range is identified as a group of medicines. The drug identification unit, for the imaging target identified as a group of drugs,
4. The medicine identification system according to claim 3, wherein the number of medicines contained in said group of medicines is calculated by counting the number of areas located within the second identification range. . 5. The drug identification system according to any one of claims 1 to 4, wherein the distance measuring sensor uses LiDAR (Light Detection and Ranging). In a drug identification method for identifying a photographed drug by performing image recognition of a photographed image of the drug,
a photographing step of photographing a drug to obtain a photographed image;
a drug identification step of identifying a drug included in the captured image;
has
detecting the thickness of the object to be photographed in the photographing step;
in the drug identification step, identifying an imaging target surrounded by an area having a thickness within a predetermined packaging identification range as a group of drugs;
The number of drugs included in the group of drugs is calculated by counting the number of regions having a thickness within a predetermined numerical identification range for the group of drugs. Method.

Images