CN116230253A - Pharmacy medicine checking method and device based on image recognition and storage medium - Google Patents
Pharmacy medicine checking method and device based on image recognition and storage medium Download PDFInfo
- Publication number
- CN116230253A CN116230253A CN202310126291.8A CN202310126291A CN116230253A CN 116230253 A CN116230253 A CN 116230253A CN 202310126291 A CN202310126291 A CN 202310126291A CN 116230253 A CN116230253 A CN 116230253A
- Authority
- CN
- China
- Prior art keywords
- medicine
- module
- image
- image recognition
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000003814 drug Substances 0.000 title claims abstract description 228
- 238000000034 method Methods 0.000 title claims abstract description 25
- 229940079593 drug Drugs 0.000 claims abstract description 74
- 230000006870 function Effects 0.000 claims description 16
- 238000012795 verification Methods 0.000 claims description 16
- 238000010586 diagram Methods 0.000 claims description 15
- 238000005457 optimization Methods 0.000 claims description 14
- 238000000605 extraction Methods 0.000 claims description 13
- 230000004927 fusion Effects 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims description 6
- 230000004931 aggregating effect Effects 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000001990 intravenous administration Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H70/00—ICT specially adapted for the handling or processing of medical references
- G16H70/40—ICT specially adapted for the handling or processing of medical references relating to drugs, e.g. their side effects or intended usage
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Pharmacology & Pharmacy (AREA)
- Biomedical Technology (AREA)
- Toxicology (AREA)
- Epidemiology (AREA)
- Public Health (AREA)
- Primary Health Care (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Chemical & Material Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medicinal Chemistry (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The invention provides a pharmacy medicine checking method, a device and a storage medium based on image recognition, which are used for acquiring an image of a target position where a medicine is located to obtain a medicine image; obtaining target medicine names and target medicine quantity from the medicine image, and matching from a preset medicine database according to the target medicine names to obtain corresponding medicine coding information; connecting a hospital information system, calling medicine inventory information corresponding to medicine coding information, matching and checking target medicine names, target medicine quantity and medicine inventory information, and obtaining an alarm signal if the checking result is inconsistent; and carrying out alarm prompt according to the alarm signal. The invention can automatically collect the medicine image, identify the names and the quantity of the target medicines to be checked from the medicine image, and automatically match and check the medicine inventory information in the hospital information system, thereby accurately allocating the medicines, reducing manual errors and ensuring the medication safety of patients.
Description
Technical Field
The invention mainly relates to the technical field of medicine supervision, in particular to a pharmacy medicine checking method and device based on image recognition and a storage medium.
Background
The pharmacy of the hospital department has strict requirements on the ex-warehouse and warehouse-in management of medicines, the pharmacy at present carries out manual verification on the allocation of medicines, and the manual allocation errors are avoided although strict management system and check flow exist, and once the errors occur, bad results are generated, so that the safety guarantee can not be provided for the product allocation work.
Disclosure of Invention
The invention aims to solve the technical problem of providing a pharmacy drug checking method, device and storage medium based on image recognition aiming at the defects of the prior art.
The technical scheme for solving the technical problems is as follows: a pharmacy medicine checking method based on image recognition comprises the following steps:
image acquisition is carried out on the target position of the medicine to obtain a medicine image;
obtaining target medicine names and target medicine quantity from the medicine image, and obtaining corresponding medicine coding information from a preset medicine database according to the target medicine names in a matching way;
connecting a hospital information system, retrieving medicine inventory information corresponding to the medicine coding information from the hospital information system, checking the target medicine names, the target medicine quantity and the medicine inventory information, and obtaining an alarm signal if the checking result is inconsistent;
and carrying out alarm prompt according to the alarm signal.
The beneficial effects of the invention are as follows: the medicine image can be automatically acquired, the names and the quantity of the target medicines to be checked are identified from the medicine image, matching check is automatically carried out on the target medicine names and the quantity of the target medicines to be checked and the medicine inventory information in the hospital information system, and an alarm is given when the check results are inconsistent, so that medicine allocation can be accurately carried out, manual errors are reduced, and the medication safety of patients is ensured.
On the basis of the technical scheme, the invention can be improved as follows.
Further, in the image recognition module, the target drug name and the target drug number are obtained from the drug image, specifically:
model optimization is carried out based on the YOLOv4 image recognition model, and an optimized YOLOv4 image recognition model is obtained;
and identifying the name of the target medicine and the quantity of the target medicine in the medicine image through the optimized YOLOv4 image identification model.
The beneficial effects of adopting the further technical scheme are as follows: the medicine can be identified through the optimized YOLOv4 image identification model, so that medicine information can be automatically obtained.
The other technical scheme for solving the technical problems is as follows: the pharmacy medicine checking device based on image recognition comprises an image acquisition module, an image recognition module, a data calling and matching module and an alarm module;
the image acquisition module is used for acquiring images of the target positions of the medicines to obtain medicine images;
the image recognition module is used for obtaining target medicine names and target medicine quantity from the medicine image, obtaining corresponding medicine coding information from a preset medicine database according to the target medicine names in a matching mode, and generating a checking instruction;
the data retrieving and checking module is used for connecting a hospital information system according to the checking instruction, retrieving medicine inventory information corresponding to the medicine coding information from the hospital information system, matching and checking the target medicine names, the target medicine quantity and the medicine inventory information, and obtaining an alarm signal if the checking result is inconsistent;
and the alarm module is used for carrying out alarm prompt according to the alarm signal.
On the basis of the technical scheme, the invention can be improved as follows.
Further, the obtaining the target drug name and the target drug quantity from the drug image specifically includes:
model optimization is carried out based on the YOLOv4 image recognition model, and an optimized YOLOv4 image recognition model is obtained;
and identifying the name of the target medicine and the quantity of the target medicine in the medicine image through the optimized YOLOv4 image identification model.
The other technical scheme for solving the technical problems is as follows: a pharmacy drug verification device based on image recognition, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, which when executed by the processor, implements the pharmacy drug verification method based on image recognition as described above.
The other technical scheme for solving the technical problems is as follows: a computer readable storage medium storing a computer program which, when executed by a processor, implements a pharmacy drug verification method based on image recognition as described above.
Drawings
Fig. 1 is a schematic flow chart of a pharmacy drug checking method according to an embodiment of the present invention;
fig. 2 is a functional block diagram of a pharmacy drug checking device according to an embodiment of the present invention.
Detailed Description
The principles and features of the present invention are described below with reference to the drawings, the examples are illustrated for the purpose of illustrating the invention and are not to be construed as limiting the scope of the invention.
Example 1:
as shown in fig. 1, the pharmacy drug checking device based on image recognition comprises an image acquisition module, an image recognition module, a data calling and matching module and an alarm module;
the image acquisition module is used for acquiring images of the target positions of the medicines to obtain medicine images;
the image recognition module is used for obtaining target medicine names and target medicine quantity from the medicine image, obtaining corresponding medicine coding information from a preset medicine database according to the target medicine names in a matching mode, and generating a checking instruction;
the data acquisition and checking module is used for connecting a hospital information system according to the checking instruction, acquiring medicine inventory information corresponding to the medicine coding information from the hospital information system, checking the target medicine names, the target medicine quantity and the medicine inventory information, and acquiring an alarm signal if the checking result is inconsistent;
and the alarm module is used for carrying out alarm prompt according to the alarm signal.
In the embodiment, the medicine images can be automatically acquired, the names and the number of the target medicines to be checked are identified from the medicine images, matching check is automatically carried out on the target medicine names and the number of the target medicines to be checked and the medicine inventory information in the hospital information system, and alarming is carried out when the check results are inconsistent, so that medicine allocation can be accurately carried out, manual errors are reduced, and the medication safety of patients is ensured.
On the basis of the above embodiment, in the image recognition module, the target drug name and the target drug number are obtained from the drug image, specifically:
model optimization is carried out based on the YOLOv4 image recognition model, and an optimized YOLOv4 image recognition model is obtained;
and identifying the name of the target medicine and the quantity of the target medicine in the medicine image through the optimized YOLOv4 image identification model.
In the above embodiment, the medicine can be identified by the optimized YOLOv4 image identification model, so that the medicine information can be automatically obtained.
Based on the above embodiment, in the image recognition module, model optimization is performed based on a YOLOv4 image recognition model, specifically:
the YOLOv4 image recognition model comprises a trunk extraction network, a characteristic network and a network loss function, wherein the output of the trunk extraction network is connected with the input of the characteristic network;
taking a Ghost module as a convolution layer of the backbone extraction network, wherein the Ghost module is used for convolving the input medicine image, outputting a characteristic diagram of the medicine image,
the calculated amount of the Ghost module convolution is as follows:
the number of parameters of the Ghost module convolution is as follows:
wherein the input characteristic diagram is h×w×c, k is the width and height of the convolution kernel, h '×w' ×n is the output characteristic diagram,
for linear transformation, x d is the size of the convolution kernel used for linear transformation;
taking a Bi-FPN module as an input feature layer of the feature network, wherein the input feature layer is used for carrying out secondary feature fusion on a feature map input by the Ghost module and weighting each input feature of a feature fusion node respectively;
specifically, the YOLOv4 image recognition model is fitted through the network loss function, wherein the network loss function comprises a prediction frame regression error, a confidence coefficient error and a classification error, the prediction frame regression error adopts CIoU loss, and the confidence coefficient error and the classification error adopt cross entropy loss.
Fitting the YOLOv4 image recognition model through the network loss function, wherein the network loss function is as follows:
where a is the weight function, v is the similarity of aspect ratios,
respectively whether the predicted grid contains this target, < +.>
And->
And respectively representing confidence and category probability of the prediction frame and the real frame, and adopting the IOU as a regression error of the prediction frame.
In the above embodiment, since the medicine identification is affected by conditions such as illumination, angle, etc., the identification accuracy can be improved by adopting the image features for identification. Meanwhile, the machine is used for replacing manual medicine identification, so that human resources can be saved, the mistaken medicine identification caused by tired staff is effectively avoided, the medicine identification accuracy is guaranteed, and the device has great promotion effects in the aspects of improving the working efficiency and the operation efficiency. And the YOLOv4 image recognition model is utilized to extract the external characteristics of the medicine such as shape, color, texture and the like, and then the characteristics are fused to effectively recognize the medicine.
On this basis, the backbone feature extraction network of YOLOv4 is redesigned using the Ghost module architecture. And introducing a Ghost module on the backbone network of the YOLOv4, wherein the Ghost module only needs to use a conventional convolution operation to obtain a part of characteristic diagrams, then obtains redundant characteristic diagrams through linear transformation, and does not need to carry out complete convolution operation. Therefore, by using some methods with lower operand to obtain redundant feature maps, floating point calculation numbers can be reduced, and the network structure can be simplified. Firstly, carrying out a traditional convolution operation on an input feature map to generate a part of feature map, wherein in the Ghost convolution process, the calculated amount of a part of channels generated first is smaller than that of standard convolution, but linear transformation is needed to compensate in order to keep the same number as the original channels. When expanding m channels, generating an identity mapping for each channel of the output feature map, and then (s- 1 ) And (3) performing linear operation, and finally enabling the number of output channels to be the same as the original number. Here a super parameter s is introduced and the size of the convolution kernel used for each linear transformation is d x d, yielding n=s x m. When the feature map is processed, the linear transformation in the Ghost module can reduce the floating point calculation amount to a certain extent, and meanwhile, the identity mapping can effectively keep the information of the feature map. Thus, the problem that some medicines with similar appearance can be identified as the same medicine and some medicines with approximate transparency can be mistakenly regarded as not exist can be solved. The medicine has the advantages of few effective pixels carried by the appearance and unobvious characteristics.
The Bi-FPN module can solve the problem that part of medicines carry few pixel points, can be used for detecting the characteristic deficiency after multiple times of rolling and downsampling, reduces the identification rate of the target medicines, and finally leads to medicine missing detection and false detection.
Specifically, a 104×104 feature output is added, and in order to enable the Bi-FPN module to adapt to YOLOv4 which is expanded into 4-scale detection, YOLOv4 simplifies 5 input feature layers in the Bi-FPN module into 4, detection is performed from four different scales of 13×13, 26×26, 52×52 and 104×104, a multi-scale feature map output in a backbone network is input into a simplified Bi-FPN structure, bottom-up secondary feature fusion is performed on the basis of PANet, and each input feature of a feature fusion node is weighted respectively. For information with different scales, the feature resolution scales are unified through up sample and down sample, then the results obtained from the bottom are subjected to feature fusion with the initial input feature map, and then new fusion is performed with the results obtained from the top and the bottom, so that the loss of the feature information is reduced.
On the basis of the foregoing embodiment, in the image recognition module, model optimization is performed based on a YOLOv4 image recognition model, and further includes:
and connecting a CA attention module behind the Ghost module, wherein the CA attention module is used for converting the channel attention of the YOLOv4 image recognition model into two one-dimensional feature codes in the vertical and horizontal directions, acquiring the features along the horizontal and vertical directions, and then aggregating the acquired features, wherein the channel outputs in the horizontal and vertical directions are as follows:
wherein H and W are coordinate values in the horizontal direction and the vertical direction respectively, each channel is coded by a pooling core with the size of (H, 1) or (1, W) according to the directions of the horizontal coordinate system and the vertical coordinate system in sequence, and x c Mapping for channel characteristics.
In the above embodiment, the CA attention module can capture not only the feature information of different channels, but also the channel information of the feature map from different directions, and then acquire the required position information. The positioning capability of the medicine can be improved, and the omission ratio of the medicine is reduced.
Example 2:
as shown in fig. 2, a pharmacy drug checking method based on image recognition includes the following steps:
image acquisition is carried out on the target position of the medicine to obtain a medicine image;
obtaining target medicine names and target medicine quantity from the medicine image, and obtaining corresponding medicine coding information from a preset medicine database according to the target medicine names in a matching way;
connecting a hospital information system, retrieving medicine inventory information corresponding to the medicine coding information from the hospital information system, checking the target medicine names, the target medicine quantity and the medicine inventory information, and obtaining an alarm signal if the checking result is inconsistent;
and carrying out alarm prompt according to the alarm signal.
On the basis of the above embodiment, the obtaining the target drug name and the target drug number from the drug image specifically includes:
model optimization is carried out based on the YOLOv4 image recognition model, and an optimized YOLOv4 image recognition model is obtained;
and identifying the name of the target medicine and the quantity of the target medicine in the medicine image through the optimized YOLOv4 image identification model.
Based on the above embodiment, the model optimization is performed based on the YOLOv4 image recognition model, specifically:
the YOLOv4 image recognition model comprises a trunk extraction network, a characteristic network and a network loss function, wherein the output of the trunk extraction network is connected with the input of the characteristic network;
taking a Ghost module as a convolution layer of the backbone extraction network, wherein the Ghost module is used for convolving the input medicine image, outputting a characteristic diagram of the medicine image,
the calculated amount of the Ghost module convolution is as follows:
the number of parameters of the Ghost module convolution is as follows:
wherein the input characteristic diagram is h×w×c, k is the width and height of the convolution kernel, h '×w' ×n is the output characteristic diagram,
for linear transformation, x d is the size of the convolution kernel used for linear transformation;
taking a Bi-FPN module as an input feature layer of the feature network, wherein the input feature layer is used for carrying out secondary feature fusion on a feature map input by the Ghost module and weighting each input feature of a feature fusion node respectively;
fitting the YOLOv4 image recognition model through the network loss function, wherein the network loss function is as follows:
where a is the weight function, v is the similarity of aspect ratios,
respectively whether the predicted grid contains this target, < +.>
And->
The confidence and class probabilities of the predicted and real frames are represented, respectively.
On the basis of the above embodiment, the model optimization based on the YOLOv4 image recognition model further includes the steps of:
and connecting a CA attention module behind the Ghost module, wherein the CA attention module is used for converting the channel attention of the YOLOv4 image recognition model into two one-dimensional feature codes in the vertical and horizontal directions, acquiring the features along the horizontal and vertical directions, and then aggregating the acquired features, wherein the channel outputs in the horizontal and vertical directions are as follows:
wherein H and W are coordinate values in the horizontal direction and the vertical direction respectively, each channel is coded by a pooling core with the size of (H, 1) or (1, W) according to the directions of the horizontal coordinate system and the vertical coordinate system in sequence, and x c Mapping for channel characteristics.
Example 3:
a pharmacy drug verification device based on image recognition, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, which when executed by the processor, implements the pharmacy drug verification method based on image recognition as described above.
Example 4:
a computer readable storage medium storing a computer program, characterized in that the pharmacy drug verification method based on image recognition as described above is implemented when the computer program is executed by a processor.
The application scene of the pharmacy medicine checking device and method based on image recognition is PIVAS intravenous medicine centralized allocation center.
The pharmacy drug checking device is used for collecting a drug image of a target position of a drug in a PIVAS intravenous drug centralized allocation center, identifying and obtaining a drug name and a drug quantity, matching and obtaining corresponding drug coding information from a preset drug database according to the drug name, retrieving drug inventory information corresponding to the drug coding information from the hospital information system, matching and checking the drug name and the drug quantity with the drug inventory information (another scene is the drug name and the drug quantity which can be matched with a drug order sheet), obtaining an alarm signal if the checking result is inconsistent, and dispensing if the checking result is inconsistent. Can accurately carry out drug allocation, reduce manual errors and ensure the medication safety of patients.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.
Claims (10)
1. The pharmacy medicine checking method based on image recognition is characterized by comprising the following steps:
image acquisition is carried out on the target position of the medicine to obtain a medicine image;
obtaining target medicine names and target medicine quantity from the medicine image, and obtaining corresponding medicine coding information from a preset medicine database according to the target medicine names in a matching way;
connecting a hospital information system, retrieving medicine inventory information corresponding to the medicine coding information from the hospital information system, matching and checking the target medicine names, the target medicine quantity and the medicine inventory information, and obtaining an alarm signal if the checking result is inconsistent;
and carrying out alarm prompt according to the alarm signal.
2. The pharmacy drug verification method according to claim 1, wherein the obtaining of the target drug name and the target drug quantity from the drug image is specifically:
model optimization is carried out based on the YOLOv4 image recognition model, and an optimized YOLOv4 image recognition model is obtained;
and identifying the name of the target medicine and the quantity of the target medicine in the medicine image through the optimized YOLOv4 image identification model.
3. The pharmacy drug verification method according to claim 2, wherein the model optimization is performed based on a YOLOv4 image recognition model, specifically:
the YOLOv4 image recognition model comprises a trunk extraction network, a characteristic network and a network loss function, wherein the output of the trunk extraction network is connected with the input of the characteristic network;
taking a Ghost module as a convolution layer of the backbone extraction network, wherein the Ghost module is used for convolving the input medicine image, outputting a characteristic diagram of the medicine image,
the calculated amount of the Ghost module convolution is as follows:
the number of parameters of the Ghost module convolution is as follows:
wherein the input characteristic diagram is h×w×c, k is the width and height of the convolution kernel, h '×w' ×n is the output characteristic diagram,
for linear transformation, x d is the size of the convolution kernel used for linear transformation;
taking a Bi-FPN module as an input feature layer of the feature network, wherein the input feature layer is used for carrying out secondary feature fusion on a feature map input by the Ghost module and weighting each input feature of a feature fusion node respectively;
fitting the YOLOv4 image recognition model through the network loss function, wherein the network loss function comprises a prediction frame regression error, a confidence coefficient error and a classification error, the prediction frame regression error adopts CIoU loss, and the confidence coefficient error and the classification error adopt cross entropy loss.
4. The pharmacy medicine verification method according to claim 3, wherein said model optimization based on YOLOv4 image recognition model further comprises the steps of:
and connecting a CA attention module behind the Ghost module, wherein the CA attention module is used for converting the channel attention of the YOLOv4 image recognition model into two one-dimensional feature codes in the vertical and horizontal directions, acquiring the features along the horizontal and vertical directions, and then aggregating the acquired features, wherein the channel outputs in the horizontal and vertical directions are as follows:
wherein H and W are respectively coordinate values in the horizontal direction and the vertical direction, each channel is coded by a pooling core with the size of (H, 1) or (1, W) according to the directions of the horizontal coordinate system and the vertical coordinate system in sequence, and x c Mapping for channel characteristics.
5. The pharmacy medicine checking device based on image recognition is characterized by comprising an image acquisition module, an image recognition module, a data calling and matching module and an alarm module;
the image acquisition module is used for acquiring images of the target positions of the medicines to obtain medicine images;
the image recognition module is used for obtaining target medicine names and target medicine quantity from the medicine image, obtaining corresponding medicine coding information from a preset medicine database according to the target medicine names in a matching mode, and generating a checking instruction;
the data retrieving and checking module is used for connecting a hospital information system according to the checking instruction, retrieving medicine inventory information corresponding to the medicine coding information from the hospital information system, matching and checking the target medicine names, the target medicine quantity and the medicine inventory information, and obtaining an alarm signal if the checking result is inconsistent;
and the alarm module is used for carrying out alarm prompt according to the alarm signal.
6. The pharmacy drug verification apparatus according to claim 5, wherein the image recognition module obtains a target drug name and a target drug quantity from the drug image, specifically:
model optimization is carried out based on the YOLOv4 image recognition model, and an optimized YOLOv4 image recognition model is obtained;
and identifying the name of the target medicine and the quantity of the target medicine in the medicine image through the optimized YOLOv4 image identification model.
7. The pharmacy drug verification device according to claim 6, wherein in the image recognition module, model optimization is performed based on a YOLOv4 image recognition model, specifically:
the YOLOv4 image recognition model comprises a trunk extraction network, a characteristic network and a network loss function, wherein the output of the trunk extraction network is connected with the input of the characteristic network;
taking a Ghost module as a convolution layer of the backbone extraction network, wherein the Ghost module is used for convolving the input medicine image, outputting a characteristic diagram of the medicine image,
the calculated amount of the Ghost module convolution is as follows:
the number of parameters of the Ghost module convolution is as follows:
wherein the input characteristic diagram is h×w×c, k is the width and height of the convolution kernel, h '×w' ×n is the output characteristic diagram,
for linear transformation, x d is the size of the convolution kernel used for linear transformation; />
Taking a Bi-FPN module as an input feature layer of the feature network, wherein the input feature layer is used for carrying out secondary feature fusion on a feature map input by the Ghost module and weighting each input feature of a feature fusion node respectively;
fitting the YOLOv4 image recognition model through the network loss function, wherein the network loss function comprises a prediction frame regression error, a confidence coefficient error and a classification error, the prediction frame regression error adopts CIoU loss, and the confidence coefficient error and the classification error adopt cross entropy loss.
8. The pharmacy drug verification apparatus according to claim 3, wherein in the image recognition module, model optimization is performed based on a YOLOv4 image recognition model, further comprising:
and connecting a CA attention module behind the Ghost module, wherein the CA attention module is used for converting the channel attention of the YOLOv4 image recognition model into two one-dimensional feature codes in the vertical and horizontal directions, acquiring the features along the horizontal and vertical directions, and then aggregating the acquired features, wherein the channel outputs in the horizontal and vertical directions are as follows:
wherein H and W are coordinate values in the horizontal direction and the vertical direction respectively, each channel is coded by a pooling core with the size of (H, 1) or (1, W) according to the directions of the horizontal coordinate system and the vertical coordinate system in sequence, and x c Mapping for channel characteristics.
9. A pharmacy drug verification device based on image recognition, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the pharmacy drug verification method based on image recognition as claimed in any one of claims 1 to 4.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the image recognition-based pharmacy drug verification method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310126291.8A CN116230253A (en) | 2023-02-16 | 2023-02-16 | Pharmacy medicine checking method and device based on image recognition and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310126291.8A CN116230253A (en) | 2023-02-16 | 2023-02-16 | Pharmacy medicine checking method and device based on image recognition and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116230253A true CN116230253A (en) | 2023-06-06 |
Family
ID=86570934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310126291.8A Pending CN116230253A (en) | 2023-02-16 | 2023-02-16 | Pharmacy medicine checking method and device based on image recognition and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116230253A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117524408A (en) * | 2023-11-30 | 2024-02-06 | 青岛大学附属医院 | AI-based intravenous administration configuration checking system and use method thereof |
-
2023
- 2023-02-16 CN CN202310126291.8A patent/CN116230253A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117524408A (en) * | 2023-11-30 | 2024-02-06 | 青岛大学附属医院 | AI-based intravenous administration configuration checking system and use method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107766894B (en) | Remote sensing image natural language generation method based on attention mechanism and deep learning | |
| CN110287960B (en) | Method for detecting and identifying curve characters in natural scene image | |
| CN108664996B (en) | Ancient character recognition method and system based on deep learning | |
| CN115063573B (en) | Multi-scale target detection method based on attention mechanism | |
| CN107944450B (en) | License plate recognition method and device | |
| US9330336B2 (en) | Systems, methods, and media for on-line boosting of a classifier | |
| CN110969245B (en) | Target detection model training method and device for medical image | |
| CN109376631B (en) | Loop detection method and device based on neural network | |
| CN112950651A (en) | Automatic delineation method of mediastinal lymph drainage area based on deep learning network | |
| CN110992325A (en) | Target counting method, device and equipment based on deep learning | |
| KR101917369B1 (en) | Method and apparatus for retrieving image using convolution neural network | |
| CN116092179A (en) | Improved Yolox fall detection system | |
| CN111275040A (en) | Positioning method and device, electronic equipment and computer readable storage medium | |
| US20240212336A1 (en) | Security check ct object recognition method and apparatus | |
| CN116230253A (en) | Pharmacy medicine checking method and device based on image recognition and storage medium | |
| CN116958958B (en) | Self-adaptive class-level object attitude estimation method based on graph convolution double-flow shape prior | |
| CN115331071A (en) | Tuberculous meningoencephalitis prediction method and system based on multi-scale feature map | |
| CN118115934A (en) | Dense pedestrian detection method and system | |
| CN115223173A (en) | Object identification method and device, electronic equipment and storage medium | |
| CN115600570A (en) | Power grid equipment early warning limit value table reconstruction method, entry method and device | |
| CN114708647A (en) | Multi-human body posture estimation detection and segmentation optimization method and device based on example segmentation | |
| CN114220082A (en) | Lane line identification method and device and computer readable storage medium | |
| CN112614107A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN117496323B (en) | Multi-scale second-order pathological image classification method and system based on transducer | |
| CN113111929B (en) | Template matching method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |