TWI782287B - Medicine placement and photography apparatus and medicine identification system - Google Patents

Abstract

The medicine placement and photography apparatus includes a housing, a medicine placement layer, a first camera module, a second camera module, a single board computer, and an output unit. The medicine placement layer, the first camera module, and the second camera module are arranged in the housing. The medicine placement layer has a top surface and a bottom surface. The first camera module faces the top surface, and the second camera module faces the bottom surface. When at least one medicine is placed on the medicine placement layer, the first camera module captures a first image of the at least one medicine and the second camera module captures a second image of the at least one medicine. The medicine identification apparatus receives the first image and the second image and inputs them into a machine learning model to determine an identify result of the least one medicine.

Description

Drug delivery and camera device and drug identification system

The invention relates to a drug delivery and photography device, a drug identification device and a drug identification system. Specifically, the present invention relates to a drug delivery and photography device comprising two camera modules, a drug identification device for drug identification using two images captured by the two camera modules, and a drug delivery and photography device comprising the aforementioned drug delivery and photography device and a drug A drug identification system for an identification device.

Many people who are hospitalized or cared for at home need to go to the pharmacy to get the medicine according to the prescription issued by the doctor. In practice, the space of most pharmacies is quite narrow, and various medicines are often placed next to each other. In addition, many drugs have highly similar names, but have completely different effects and appearances. Therefore, when a pharmacist is faced with a huge workload, it is easy to make medication errors. In the event of medication errors, not only will people not be cured, but it will also lead to medical disputes.

In order to solve the above problems, one method currently adopted in the industry is to capture an image of the drug through a camera lens, generate a feature vector of the drug image, and then compare the feature vector with other feature vectors in the database to determine the type of drug. General medicine includes two types of tablets and capsules. Usually, the drug information of the tablet is only contained on one side, while the other side is plain. The drug information of the capsule is contained on the outside, but its shape is in the middle of two hemispheres. Clip a cylindrical, easy to roll. Regardless of the type of drug, if the drug information is not intentionally directed toward the camera lens when placing the drug, the camera lens will When shooting, it is often difficult to clearly obtain complete drug information images. In addition, most of the current drug identification devices in the industry are inconvenient to move, and it is difficult for users to easily place and take out the drugs, especially when users need to operate such drug identification devices in a narrow working environment, which is more inconvenient and error-prone.

In view of this, how to improve the above-mentioned problem of drug identification and provide a drug identification technology that can greatly reduce the probability of drug administration errors and minimally affect the drug administration process is an urgent problem to be solved in the technical field of the present invention.

One object of the present invention is to provide a drug delivery and photography device. The medicine injecting and photographing device comprises a casing, a medicine layer, a first photographing module, a second photographing module, a single-board computer and an output unit. Wherein the drug-containing layer is arranged in the casing, is formed of a light-transmitting material, and has an upper surface and a lower surface. The first camera module is disposed in the casing and faces the upper surface. The second camera module is disposed in the casing and faces the lower surface. The single board computer is electrically connected to the first camera module and the second camera module, wherein when at least one drug is placed on the drug layer, the single board computer controls the first camera module to take pictures Generate a first image, control the second camera module to take pictures to generate a second image, and send the first image and the second image to a medicine identification device, so that the medicine identification device can use the first image And the second image determines an identification result of the at least one medicine, wherein the single board computer also receives the identification result from the medicine identification device. The output unit is electrically connected to the single board computer, and outputs a signal related to the identification result.

Another object of the present invention is to provide a medicine identification device. The medicine identification device includes a transceiver interface and a processor. Wherein the transceiver interface receives a first image and a second image of at least one drug from a drug delivery and camera device. The processor is electrically connected to the transceiver interface, and the second An image and the second image are input into a machine learning model to determine an identification result of the at least one drug. Wherein the transceiving interface also transmits the identification result to the drug delivery and photography device.

Another object of the present invention is to provide a drug identification system. Wherein the drug identification system includes the operations performed by the drug injection and photography device and the drug identification device.

The drug identification technology provided by the present invention (including at least drug delivery and photography device, drug identification device and drug identification system) uses the drug delivery and photography device to obtain two images of at least one drug to be inspected, and then uses the drug identification The device performs drug identification on the at least one drug based on the two images. In summary, when at least one drug is placed on the upper surface of the drug layer of the drug delivery and photography device, the drug delivery and photography device generates a first image and a second image of the at least one medicine, and the The first image and the second image are sent to the medicine identification device. The medicine identification device then inputs the first image and the second image into a machine learning model to determine an identification result of the at least one medicine. The medicine identification technology provided by the present invention can output a signal related to the identification result through an output unit.

Since the drug identification technology provided by the present invention is to identify the upper surface and the lower surface of at least one drug through the first camera module facing the upper surface of the drug layer and the second camera module facing the lower surface of the drug layer By taking pictures, the complete image information of the at least one drug can be obtained, so as to achieve accurate and stable identification results. The drug delivery and photography device provided by the present invention is easy to operate (the user only needs to throw the medicine into the drug delivery and photography device 1 ), and is easy to carry. Therefore, the user can move the drug dispensing and photographing device 1 arbitrarily in the working environment, which is convenient for the user to identify the drug in different places, and can meet the working needs of the user in a narrow space such as a pharmacy.

The detailed technology and implementation methods of the present invention are described below in conjunction with the drawings, so that those with ordinary knowledge in the technical field of the present invention can understand the technical characteristics of the claimed invention.

1: Drug delivery and photography device

3: Drug identification device

11: shell

11a: Upper shell

11b: Lower shell

11c: Upper opening

11d: lower opening

13: drug layer

13a: upper surface

13b: lower surface

15, 17: Photography module

19:Single board computer

21: Output unit

23, 25: Light source module

27: Input interface

28: track

29: Shaft

31: Sending and receiving interface

33: Processor

40: Identification result

I1, I2: Image

Figure 1A depicts a three-dimensional schematic diagram of a drug delivery and photography device 1 according to an embodiment; Figure 1B depicts an exploded view of a drug delivery and photography device 1 according to an embodiment; Figure 1C depicts a drug delivery and photography device 1 according to an embodiment Figure 2A depicts a side perspective view of the drug delivery track 28 of some embodiments of the drug delivery and photography device 1; Figure 2B depicts the side perspective view of the drug delivery and photography device 1 of some embodiments. A schematic diagram of the rotating shaft 29; FIG. 2C depicts a schematic diagram of the operation of the rotating shaft 29; FIG. 2D depicts a perspective view of a drug delivery and camera device 1 in certain embodiments; and FIG. 3 depicts a schematic structural view of a drug identification device 3 in an embodiment .

The following will explain the drug delivery and photography device, the drug identification device and the drug identification system provided by the present invention through the embodiments. However, these embodiments are not intended to limit the present invention to be implemented in any environment, application or manner as described in these embodiments. Therefore, the description of the following embodiments is only for the purpose of illustrating the present invention, rather than limiting the scope of the present invention. It should be understood that in the following embodiments and drawings, elements not directly related to the present invention have been omitted and not shown, and the size of each element and the size ratio between elements in the drawings are only for illustration and description, and It is not intended to limit the scope of the present invention.

One embodiment of the present invention is a drug identification system, which includes a drug delivery and photography device 1 (see Figure 1A, Figure 1B and Figure 1C) and a drug identification device 3 (see Figure 3), In addition, the drug delivery and photography device 1 is used in conjunction with the drug identification device 3 . The drug identification system uses the drug injection and photography device 1 to obtain two images of at least one drug, and transmits the two images to the drug identification device 3, so that the drug identification device 3 determines the at least one drug according to the two images. An identification result (it can be the name of each of the at least one medicine, which medicines and how many medicines there are, or whether it is administered correctly). The structure and operation mechanism of the drug delivery and imaging device 1 and the drug identification device 3 will be described in detail below.

Fig. 1A, Fig. 1B and Fig. 1C respectively depict a three-dimensional schematic diagram, an exploded diagram and a side schematic diagram of the drug delivery and photography device 1. Specifically, the drug delivery and camera device 1 includes a casing 11 , a drug layer 13 , two camera modules 15 , 17 , a single board computer 19 and an output unit 21 . In this embodiment, the housing 11 includes an upper housing 11a and a lower housing 11b, and the upper housing 11a and the lower housing 11b are detachable, so that the internal modules/components of the housing 11 can be disassembled. In other embodiments, the housing 11 can adopt other structures and designs, but the user should be able to open the housing 11 to disassemble the internal modules/components. In addition, the present invention does not limit the shape, size and material of the casing 11 . For example, the casing 11 can be a cylinder, and can be of a size that allows the user to carry it easily.

The medicine layer 13 is arranged inside the casing 11, and is formed by a light-transmitting material, and its shape (for example: plate shape, disc shape, disc shape) and size depend on the inner space of the casing 11, but it needs to be able to place at least one drug. The drug-loaded layer 13 has at least two surfaces, and when the drug-loaded layer 13 is placed inside the housing 11 , the two surfaces are respectively used as an upper surface 13 a and a lower surface 13 b. In some embodiments, the upper surface 13a of the drug placement layer 13 can be a concave surface, so that when at least one drug is placed on the upper surface 13a, it can be concentrated and distributed at the center of the upper surface 13a.

The camera modules 15 and 17 are also disposed inside the casing 11 . In this embodiment, since the casing 11 includes the upper casing 11a and the lower casing 11b, the camera module 15 can be arranged inside the upper casing 11a, and the camera module 17 can be arranged in the lower casing 11b internal. It should be explained that the camera modules 15 and 17 each have There is a lens, the camera module 15 faces the upper surface 13a of the drug layer 13, and the camera module 17 faces the lower surface 13b of the drug layer 13. The present invention does not limit the distance between the camera module 15 and the drug-containing layer 13 , and does not limit the shooting angle of the camera module 15 , as long as the upper surface 13 a of the drug-containing layer 13 can be photographed. Similarly, the present invention does not limit the distance between the camera module 17 and the drug-containing layer 13 , and does not limit the shooting angle of the camera module 17 , as long as the lower surface 13 b of the drug-containing layer 13 can be photographed.

The present invention does not limit the location where the single board computer 19 is installed. The single board computer 19 can be disposed inside the casing 11 (for example: the bottom inside the casing 11 , inside the upper casing 11 a ) or outside. Although the present invention does not limit the location of the single-board computer 19 , the single-board computer 19 is electrically connected to the camera modules 15 , 17 and the output unit 21 . The single board computer 19 is a printed circuit board integrating a microprocessor (or other various processors, central processing unit, digital signal processor), a storage, an input and output interface and a power module. In other embodiments, if the single-board computer 19 does not include a power module, the drug administration and photography device 1 needs to include a power module to provide power to the camera modules 15 , 17 , the single-board computer 19 and the output unit 21 .

In addition, the present invention does not limit the location of the output unit 21 . Similarly, the output unit 21 may be disposed inside or outside the casing 11 . The output unit 21 can be a display (for example: Light-Emitting Diode (Light-Emitting Diode; LED) display, liquid crystal display (Liquid Crystal Display; LCD), but not limited thereto), a light signal, a speaker or/ and a wireless transceiver, but not limited thereto. It should be understood that the type of output unit 21 affects where it is placed. For example, if the output unit 21 is a display, it must be set at a position where the user can watch the contents of the display. For another example, if the output unit 21 is a speaker, it can be disposed inside or outside the housing 11 , but the user should be able to hear the sound from the speaker.

In this embodiment, when at least one drug is placed on the upper surface 13a of the drug-containing layer 13, the single-board computer 19 controls the camera module 15 to take pictures to generate an image I1, and then controls the camera module 17 to take pictures. to generate another image I2. It should be noted that the single board computer 19 will control the camera module 15 and the camera module 17 to take pictures at different time points to avoid mutual interference. In addition, the present invention does not limit the shooting sequence of the camera modules 15 and 17; in other words, the single board computer 19 can first control the camera module 17 to take pictures, and then control the camera module 15 to take pictures.

In this embodiment, in order to facilitate the user to place medicine on the upper surface 13 a of the medicine placement layer 13 , the casing 11 also has an upper opening 11 c located above the medicine placement layer 13 . In this embodiment, since the housing 11 includes the upper housing 11a and the lower housing 11b, the upper opening 11c can be formed on the upper housing 11a (for example: the top surface of the upper housing 11a, the side surface of the upper housing 11a ). When the user throws the medicine into the medicine dispensing and photographing device 1 through the upper opening 11c, the medicine will drop to the medicine-containing layer 13 . In some embodiments, in order to allow the drug injected from the upper opening 11c to reach the upper surface 13a of the drug-containing layer 13 smoothly, a track 28 may be provided below the upper opening 11c, as shown in FIG. 2A. When the user throws the medicine into the medicine dispensing and photographing device 1 through the upper opening 11 c, the medicine will slide to the upper surface 13 a of the medicine-containing layer 13 through the track 28 .

In this embodiment, in order to facilitate the user to take out the medicine, the casing 11 also has a lower opening 11 d, and the lower opening 11 d is located below the medicine-containing layer 13 . In this embodiment, since the casing 11 includes the upper casing 11a and the lower casing 11b, the lower opening 11d can be formed in the lower casing 11b (for example: the bottom surface of the lower casing 11b, the side surface of the lower casing 11b) . In some embodiments, the drug delivery and photography device 1 further includes a rotating shaft 29 , and the rotating shaft 29 is embedded in the drug placement layer 13 , as shown in FIG. 2B . When the user rotates the rotating shaft 29, the drug on the drug layer 13 will drop out of the casing 11 from the lower opening 11d, as shown in FIG. 2C. The user can cover the lower opening 11d with the medicine bag, so that the medicine can be directly dropped into the medicine bag.

It should be noted that, in some embodiments, if the convenience of placing medicines is not considered, the upper opening 11c may not be formed on the casing 11 . Similarly, in some embodiments, if the convenience of taking out the medicine is not considered, the lower opening 11d may not be formed on the casing 11, and the rotating shaft 29 may not be provided.

In some embodiments, in order to obtain better quality images, the casing 11 of the drug delivery and photography device 1 is formed of an opaque material, and the drug delivery and photography device 1 further includes two light source modules 23, 25, As shown in Figure 2D. The light source modules 23 and 25 can be various lighting devices (for example: LED lamps) capable of lighting. The light source module 23 is disposed inside the casing 11 and above the upper surface 13 a of the drug-containing layer 13 for illuminating the upper surface 13 a of the drug-containing layer 13 . The light source module 25 is disposed inside the casing 11 and is located below the lower surface 13b of the drug-containing layer 13 for illuminating the lower surface 13b of the drug-containing layer 13 . In order to achieve a better lighting effect, both the light source modules 23 and 25 can be disposed in the casing 11 in a surrounding manner.

Both the light source modules 23 and 25 are electrically connected to the single board computer 19 . The single board computer 19 can control the light source modules 23 and 25 to perform lighting at different time points respectively. Specifically, when the single-board computer 19 controls the camera module 15 to take pictures, the light source module 23 will be activated to illuminate the upper surface 13a of the medicine layer 13, and at this time the camera module 17 and the light source module 25 are turned off. . When the single board computer 19 controls the camera module 17 to take pictures, the light source module 25 will be activated to illuminate the lower surface 13b of the medicine layer 13, and the camera module 15 and the light source module 23 are turned off.

In this embodiment, after the single-board computer 19 of the drug injection and photographing device 1 obtains the images I1 and I2 of at least one drug placed on the upper surface 13a of the drug-containing layer 13, the single-board computer 19 will display the images I1 and I2 Send to the medicine identification device 3 for identification.

For ease of understanding, please refer to FIG. 3 , which is a schematic diagram depicting the structure of the medicine identification device 3 . In detail, the medicine identification device 3 includes a transceiver interface 31 and a processor 33 , wherein the processor 33 is electrically connected to the transceiver interface 31 . The transceiver interface 31 is any interface capable of sending and receiving data with the drug delivery and photography device 1 , such as a wireless transceiver interface. The processor 33 can be various processors, central processing units, microprocessors or other computing devices known to those skilled in the art to which the present invention pertains.

The medicine identification device 3 obtains the images I1 and I2 through the transceiver interface 31 . Next, processor 33 Inputting the image I1 and the image I2 into a trained machine learning model to determine an identification result of the at least one drug. The machine learning model can be a convolutional neural network (Convolutional Neural Network; CNN) model, a recurrent neural network (Recurrent Neural Network; RNN), but not limited thereto. Those with ordinary knowledge in the technical field to which the present invention belongs are well-known for training methods of various machine learning models, so they will not be described in detail.

For example, the information of various medicines stored in a medicine database (such as the name, color, shape and size of medicines) can be used to train the machine learning model. After the processor 33 inputs the images I1 and I2 into the machine learning model, the machine learning model will recognize that the image I1 contains at least one first object, and recognize the color, shape and size of each of the first objects. In addition, the machine learning model recognizes that the image I2 includes at least one second object, and recognizes the color, shape and size of each of the second objects. It should be noted that the processor 33 can calculate the size of each of the first objects and the size of each of the second objects according to the distances between the camera modules 15 , 17 and the drug-containing layer 13 . The machine learning model determines an identification result 40 of the at least one drug according to the color, shape and size of each of the first objects and each of the second objects. For example, the identification result 40 may be the drug name of each of the at least one drug. For another example, the identification result 40 may be which kinds of medicines there are, and how many pills of each kind of medicines there are.

Subsequently, the transceiving interface 31 of the drug identification device 3 transmits the identification result 40 to the drug delivery and photography device 1 , and the single board computer 19 of the drug delivery and photography device 1 receives the identification result 40 . The output unit 21 of the drug delivery and photography device 1 can output a signal related to the recognition result 40 . For example, the output unit 21 can output (for example: display, make a sound) the recognized drug name of each of the at least one drug. For another example, the output unit 21 can output (for example: display, make a sound) which kinds of medicines there are, and how many medicines there are.

In some embodiments, the drug recognition system can use other content as the recognition result 40 . Specifically, the drug delivery and photography device 1 further includes an input interface 27 , and the input interface 27 is electrically connected to the single board computer 19 . The drug delivery and camera device 1 can obtain expected drug information through the input interface 27 . For example, the input interface 27 can be a barcode scanner, and the user can scan a barcode of a medicine bag through the input interface 27 to obtain expected drug information. It should be noted that the medicine bag barcodes can be one-dimensional barcodes (Barcodes) or two-dimensional barcodes (QR-codes).

In these embodiments, the single-board computer 19 can also transmit the expected drug information to the drug identification device 3 . The drug identification device 3 receives the drug information from the drug delivery and photography device 1 through the transceiver interface 31 . When the processor 33 uses the machine learning model to identify the medicine on the images I1 and I2 , it also refers to the medicine information to determine the medicine recognition result 40 . Specifically, the machine learning model first identifies the drug name of each of the at least one drug (or which drugs are there, and how many drugs are there in each drug), and then uses the drug information to determine the identification result 40 . In other words, the recognition result 40 at this time is whether the medication is correct, and the output unit 21 can display whether the medication is correct or wrong according to the recognition result 40 . If the identification result 40 is a medication error, the output unit 21 can also generate a warning signal.

In some embodiments, the single board computer 19 does not transmit the expected drug information to the drug identification device 3 . The identification result 40 received by the single board computer 19 is the name of each of the at least one drug identified by the machine learning model (or which drugs are there, and how many drugs are there in each drug). The single board computer 19 compares the expected drug information with the received identification result 40 , and the output unit 21 then outputs a signal whether the drug is correct or wrong according to the comparison result, which is not repeated here.

To sum up, the drug identification system provided by the present invention obtains two images of at least one drug to be inspected by the drug delivery and photography device 1, and then the drug identification device 3 performs an inspection of the at least one drug based on the two images. identify. The drug delivery and photography device 1 provided by the present invention can allow users to Place at least one drug to be identified on the drug-containing layer 13 (for example, throw the drug into the drug delivery and photography device 1, and the drug will automatically fall to the drug-containing layer 13). When the at least one drug is placed on the drug layer 13 , the single board computer 19 controls the camera module 15 to take pictures to generate the image I1 , and controls the camera module 17 to take pictures to generate the image I2 . Since the camera modules 15 and 17 respectively face the upper surface 13 a and the lower surface 13 b of the medicine layer 13 , complete image information of each of the at least one medicine can be obtained. The medicine identification device 3 obtains the images I1 and I2 and inputs them into a machine learning model to determine an identification result of the at least one medicine. Since the images I1 and I2 cover the complete image information of each of the at least one drug, the machine learning model can make accurate identification results.

In addition to providing accurate and stable identification results, the drug delivery and imaging device 1 provided by the present invention is easy to operate (the user only needs to throw the drug into the drug delivery and imaging device 1 ), and is easy to carry. Therefore, the user can move the drug dispensing and photographing device 1 arbitrarily in the working environment, which is convenient for the user to identify the drug in different places, and can meet the working needs of the user in a narrow space such as a pharmacy.

The above embodiments are only used to exemplify some implementations of the present invention and explain the technical features of the present invention, rather than to limit the scope and scope of the present invention. Any changes or equivalence arrangements that can be easily accomplished by those with ordinary knowledge in the technical field of the present invention belong to the scope claimed by the present invention, and the scope of protection of the rights of the present invention is subject to the scope of the patent application.

1: Drug delivery and photography device

11: shell

11c: Upper opening

11d: lower opening

13: drug layer

15, 17: Photography module

19:Single board computer

23, 25: Light source module

28: track

29: Shaft

Claims (10)

A drug injection and photographing device, comprising: a casing with an upper opening and a lower opening, wherein a track is arranged below the upper opening; a drug layer is arranged in the casing and is formed of a light-transmitting material, and It has an upper surface and a lower surface; a first camera module is arranged in the casing and faces the upper surface; a second camera module is arranged in the casing and faces the lower surface; a single A board computer is electrically connected to the first camera module and the second camera module, wherein at least one drug is suitable for being injected from the upper opening and slid to the upper surface of the drug-containing layer through the track, when the drug-containing When at least one drug is placed on the upper surface of the layer, the single-board computer controls the first camera module to shoot to generate a first image, controls the second camera module to shoot to generate a second image, and The first image and the second image are sent to a drug identification device, so that the drug identification device determines an identification result of the at least one drug according to the first image and the second image, wherein the single board computer also obtains from the drug The identification device receives the identification result; an output unit is electrically connected to the single-board computer, and outputs a signal related to the identification result; and a rotating shaft is embedded with the medicine layer so that a user can rotate the rotating shaft The at least one drug on the drug-loaded layer falls out of the casing from the lower opening. The drug delivery and photography device as described in claim 1, further comprising: a first light source module, electrically connected to the single-board computer, and disposed in the casing above the upper surface of the drug-containing layer; and a second light source module, electrically connected to the single-board computer, and arranged in the casing at the position below the lower surface of the drug layer; wherein, when the single-board computer controls the first camera module to take pictures, it also starts the first light source module to illuminate, and when the single-board computer controls the second camera module When the group takes pictures, the second light source module is also activated for lighting. The medicine dispensing and photographing device as described in claim 2, wherein the first light source module is arranged around the casing, and the second light source module is arranged around the casing. The medicine dispensing and photographing device as described in claim 1 further includes: an input interface electrically connected to the single board computer and obtaining medicine information. The drug dispensing and photographing device as described in claim 4, wherein the input interface is a barcode scanner, and the input interface obtains the drug information after scanning the barcode of a medicine bag. The drug delivery and photography device as claimed in claim 1, wherein the housing is opaque. The medicine dispensing and photographing device according to claim 1, wherein the housing includes an upper housing and a lower housing. The drug dispensing and photographing device as described in Claim 1, wherein the upper surface of the drug placement layer is a concave surface. The medicine dispensing and photographing device as described in Claim 1, wherein the single board computer controls the first photographing module and the second photographing module to take photographs at different time points. A drug identification system, comprising: a drug delivery and photography device as described in claim 1; and a drug identification device, which receives at least a first image and a second image of a drug from the drug delivery and photography device, and the first image An image and the second image are input into a machine learning model to determine a recognition result of the at least one drug, and the recognition result is sent to the drug delivery and camera device.

Images