CN115620305A - Photographing and medicine recognizing system based on deep learning and using method thereof - Google Patents
Photographing and medicine recognizing system based on deep learning and using method thereof Download PDFInfo
- Publication number
- CN115620305A CN115620305A CN202211279276.9A CN202211279276A CN115620305A CN 115620305 A CN115620305 A CN 115620305A CN 202211279276 A CN202211279276 A CN 202211279276A CN 115620305 A CN115620305 A CN 115620305A
- Authority
- CN
- China
- Prior art keywords
- medicine
- module
- photographing
- image
- identified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/19173—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/24—Character recognition characterised by the processing or recognition method
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a photographing and medicine recognizing system based on deep learning and a using method thereof, and belongs to the technical field of medicine recognition. A photographing and medicine recognizing system based on deep learning comprises a camera shooting module, an image preprocessing module, an image segmentation module, a target detection module and an OCR recognition module. The photographing medicine identification system based on deep learning is convenient to operate, efficient and accurate, an object to be detected can be identified and distinguished through photographing, medicine information and a medicine specification can be obtained, and the phenomenon that the old is deceived by false advertisements and health care products are mistakenly used as medicines is effectively prevented.
Description
Technical Field
The invention belongs to the technical field of medicine identification, and particularly relates to a photographing medicine identification system based on deep learning and a use method thereof.
Background
In recent years, the market scale of the health care product industry in China is enlarged year by year, and the market demand is strong. In the age of the health care products, some manufacturers have been publicized excessively, so that the discrimination capability of people on the health care products and the medicines is lower and lower. Especially for middle-aged and elderly people, the health care products are often mistakenly used as medicines through exaggerated introduction of sales personnel or advertisements and other channels.
Aiming at the problems, a medication assistant, namely a clove garden, appears on the market at present, but the clove garden only has the inquiry function, cannot directly take pictures to identify the medicines, is inconvenient to use, and is particularly difficult to distinguish and inquire by the old due to some imported medicines. Although the Taobao and the Jingdong have the functions of photographing and identifying the article, the identification effect is poor, the medicine resource information is incomplete, the health-care product and the medicine cannot be accurately distinguished, only the primary article identifying function is provided, and the medicine information and the medicine specification cannot be further obtained.
Therefore, a tool for helping a user to identify a medicine efficiently and accurately is needed.
Disclosure of Invention
In view of the above, the present invention provides a photographing and medicine identification system based on deep learning and a method for using the same. The invention aims to solve the problem that the existing medicine identification system cannot efficiently and accurately identify medicines and distinguish medicines, health care products and medical instruments.
In order to achieve the above object, the present invention provides a photographing and medicine identification system based on deep learning, comprising:
the camera module is used for shooting and recording the medicines, health care products and medical instruments to be identified;
the image preprocessing module is used for preprocessing the digital image shot by the camera module;
the image segmentation module is used for carrying out foreground-background segmentation on the preprocessed digital image so as to extract a foreground main body;
the target detection module is used for carrying out target detection on the foreground main body;
a detection model is arranged in the target detection module, and the detection model can classify the foreground main body into medicines, health products, medical instruments or the like;
the OCR recognition module is used for performing OCR recognition on the object to be recognized which is judged to be 'medicine' or 'other', so as to obtain medicine related information of the object to be recognized;
the OCR recognition module is associated with the database and can further acquire the drug information and the drug specification.
OCR (Optical Character Recognition) refers to a process of scanning text data, analyzing an image file, and acquiring text and layout information.
Further, the preprocessing comprises image graying, filtering and denoising, sharpening and correcting.
Further, the detection model is constructed based on training data of a drug feature database.
Further, the training data includes product images and feature images of drugs, health products, medical instruments.
Further, the feature images include the following three categories:
medicine characteristic image: electronic supervision codes, class A OTC marks and class B OTC marks;
health product characteristic image: blue cap sub-logo;
medical instrument feature image: various instrument flags.
Further, the drug related information includes a drug name, an approval document number, and a registration number.
The invention also provides a use method of the photographing and medicine identifying system based on deep learning, which comprises the following steps:
s1, constructing a detection model through a medicine characteristic database;
s2, shooting an object to be identified by using a camera module;
s3, preprocessing the image shot in the step S2;
s4, performing foreground and background segmentation on the digital image preprocessed in the step S3 to obtain a foreground main body;
s5, carrying out target detection on the foreground main body through the detection model of deep learning, and classifying the foreground main body: pharmaceuticals, nutraceuticals, medical devices, or the like;
if the foreground main body of the object to be recognized is judged to be a medicine or other objects, performing OCR recognition to recognize medicine related information in an OCR text;
if the foreground main body of the object to be identified is judged to be a health-care product or a medical instrument, stopping, and exporting a judgment result;
s6, inquiring a yz _ inststruct table according to the medicine related information identified in the step S5, and deriving a final result, wherein the following three conditions exist:
A. inquiring the medicine and having a corresponding medicine specification;
B. inquiring the medicine without the corresponding medicine specification;
C. the query result is null, indicating that it is not a drug.
The invention provides a photographing and medicine recognizing system based on an OCR recognition technology, which comprises:
the camera module is used for shooting and recording the medicines, health care products and medical instruments to be identified;
the OCR module is used for extracting character information of the picture shot by the camera module;
and the keyword matching module is used for matching the character information extracted by the OCR module so as to classify the pictures shot by the camera module into medicines, health-care products, medical instruments or the like.
The invention also provides a using method of the photographing and medicine recognizing system based on the OCR recognition technology, which comprises the following steps:
s1, shooting an object to be identified by using a camera module;
s2, extracting character information of the picture shot by the camera module by using an OCR recognition module;
s3, matching the character information extracted by the OCR module by using a keyword matching module so as to classify the object to be recognized: pharmaceuticals, nutraceuticals, medical devices, or the like;
if the output matching result of the object to be identified is a medicine or other objects, inquiring a yz _ instruct table and deriving a final result;
and if the output matching result of the object to be identified is a health-care product or a medical instrument, stopping outputting the judgment result.
The invention has the beneficial effects that:
1. the invention provides a photographing medicine identification system based on deep learning, which is more convenient to use, objects to be detected can be identified and distinguished by photographing, and if the objects to be detected are medicines, related information such as medicine specifications can be obtained, so that the old people can be effectively prevented from being deceived by false advertisements, and health care products can be mistakenly used as the medicines.
2. The invention provides a photographing and medicine identifying system based on deep learning, which is provided with a target detection module and a detection model, wherein the detection model is based on a medicine characteristic database, and related medicines are wider and more comprehensive and have stronger specialty; the invention is also provided with an image preprocessing module and an image segmentation module, so that the product image and the characteristic image extracted from the object to be identified are clearer and combined, the medicine identification is more efficient and accurate, and medicines, health care products and medical instruments can be distinguished.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
FIG. 1 is a flow chart of a photographing and medicine identification system based on deep learning according to the present invention;
FIG. 2 is a functional block diagram of a photographing and medicine recognition system based on deep learning according to the present invention;
FIG. 3 is a flow chart of a photographing and medicine recognition system based on OCR recognition technology according to the present invention;
FIG. 4 is a functional block diagram of a photographing and medicine recognition system based on OCR recognition technology according to the present invention;
FIG. 5 is a diagram of the package of Fenbid in example 1;
FIG. 6 is a diagram of the packaging of Chinese nutritionals of example 2;
fig. 7 is a diagram of the package of the oral examination bag of example 3.
Detailed Description
In order to make the technical solutions, advantages and objects of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the invention without any inventive step, are within the scope of protection of the present application.
As shown in fig. 1 and fig. 2, the present invention provides a photographing and medicine recognition system based on deep learning, which includes a camera module, an image preprocessing module, an image segmentation module, a target detection module, and an OCR recognition module.
And the camera module is used for shooting and recording the medicines, health-care products and medical instruments to be identified.
And the image preprocessing module is used for preprocessing the digital image shot by the camera module, and the preprocessing comprises image graying, filtering and denoising, sharpening and correcting.
The image segmentation module is used for carrying out foreground and background segmentation on the preprocessed digital image so as to extract a foreground main body;
the target detection module is used for carrying out target detection on the foreground main body, and the target detection module can classify the foreground main body into medicines, health-care products, medical appliances or the like;
a detection model is arranged in the target detection module and is constructed based on training data of a medicine characteristic database. The training data comprises product images and characteristic images of medicines, health products and medical instruments, and the characteristic images comprise the following three types:
medicine characteristic image: electronic supervision codes, class A OTC marks and class B OTC marks;
health product characteristic image: a blue cap logo;
medical instrument feature image: various instrument flags.
And the OCR recognition module is used for performing OCR recognition on the object to be recognized which is judged to be 'medicine' or 'other', so as to obtain medicine related information of the object to be recognized, wherein the medicine related information comprises a medicine name, an approval document number and a registration certificate number.
The OCR module is associated with a database, the database is a yz _ inststruct table in the embodiment, and the medicine information and the medicine specification can be further obtained according to the yz _ inststruct table.
Example 1
In the embodiment, a photographing and medicine identification system based on deep learning is used for identifying the Fenbid, and the identification process is as follows:
shooting a packing box of the Fenbid by using a camera module, wherein the shot image is shown in figure 5, and then preprocessing the shot image; after the preprocessing is finished, performing foreground and background segmentation on the digital image to obtain a foreground main body; then, target detection is carried out on the foreground main body through a deep learning detection model, the Fenbid can be classified into medicines, and characters such as Chinese medicine standards and indications are contained in the graph 5; then performing OCR recognition to recognize medicine related information in the OCR text; and then, inquiring a yz _ inststruct table, and deriving the medicine information and a corresponding medicine specification.
Example 2
The embodiment uses a photographing medicine identification system based on deep learning to identify national nutritions, and the identification process is as follows:
shooting the packaging box of the national nutritionals by using a camera module, wherein the shot image is as shown in figure 6, and then preprocessing the shot image; after the preprocessing is finished, performing foreground and background segmentation on the digital image to obtain a foreground main body; then, target detection is carried out on the foreground main body through a detection model of deep learning, national total nutrient can be classified into health products, and the characters such as 'health food', 'Weishi health word' and the like are contained in the graph 6; and then deriving a judgment result.
As shown in fig. 3 and 4, the present invention provides a photographing and medicine recognition system based on OCR recognition technology, comprising:
the camera module is used for shooting and recording the medicines, health care products and medical instruments to be identified;
the OCR recognition module is used for extracting character information of the picture shot by the camera module;
and the keyword matching module is used for matching the character information extracted by the OCR recognition module so as to classify the pictures shot by the camera module into medicines, health-care products, medical instruments or the like.
Example 3
In this embodiment, the photographing and medicine-identifying system based on deep learning is used to identify the oral cavity examination package, and the identification process is as follows:
shooting the package of the oral cavity inspection package by using a camera module, wherein the shot image is as shown in figure 7, and then extracting character information in the picture shot by the camera module by using an OCR recognition module; then, a keyword matching module is used for matching the character information extracted by the OCR recognition module, so that the objects to be recognized are classified, the oral cavity examination package can be classified into medical instruments, and the characters such as 'food and drug supervision instruments' are contained in the figure 7; and then deriving the judgment result.
When the photographing and medicine recognizing system is used for recognizing the medicine, the efficiency is high, the object to be detected can be quickly distinguished, the medicine information and the medicine specification can be obtained, and the phenomenon that the old is deceived by false advertisements and the health care product is mistakenly used as the medicine can be effectively prevented.
Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered in the protection scope of the present invention.
Claims (9)
1. A photographing and medicine recognizing system based on deep learning is characterized by comprising:
the camera module is used for shooting and recording the medicines, health care products and medical instruments to be identified;
the image preprocessing module is used for preprocessing the digital image shot by the camera module;
the image segmentation module is used for carrying out foreground-background segmentation on the preprocessed digital image so as to extract a foreground main body;
the target detection module is used for carrying out target detection on the foreground main body;
a detection model is arranged in the target detection module, and the detection model can classify the foreground main body into medicines, health-care products, medical instruments or the like;
the OCR recognition module is used for performing OCR recognition on the object to be recognized which is judged to be 'medicine' or 'other', so as to obtain medicine related information of the object to be recognized;
the OCR recognition module is associated with the database and can further acquire the drug information and the drug specification.
2. The system of claim 1, wherein the system comprises: the preprocessing comprises image graying, filtering and denoising, sharpening and correcting.
3. The system of claim 1, wherein the system comprises: the detection model is constructed based on training data of a drug feature database.
4. The deep learning based photo-taking and drug-identifying system of claim 3: the training data comprises product images and characteristic images of medicines, health products and medical instruments.
5. The system of claim 4, wherein the feature images include the following three categories:
medicine characteristic image: electronic supervision codes, class A OTC marks and class B OTC marks;
health product characteristic image: blue cap sub-logo;
medical instrument feature image: various instrument markers.
6. The system of claim 1, wherein the system comprises: the drug related information includes a drug name, an approval document number, and a registration number.
7. A use method of a photographing and medicine recognizing system based on deep learning is characterized by comprising the following steps:
s1, constructing a detection model through a medicine characteristic database;
s2, shooting an object to be identified by using a camera module;
s3, preprocessing the image shot in the step S2;
s4, performing foreground and background segmentation on the digital image preprocessed in the step S3 to obtain a foreground main body;
s5, carrying out target detection on the foreground main body through the detection model of deep learning, and classifying the foreground main body: pharmaceuticals, nutraceuticals, medical devices, or the like;
if the foreground main body of the object to be recognized is judged to be a medicine or other objects, performing OCR recognition to recognize medicine related information in an OCR text;
if the foreground main body of the object to be identified is judged to be a health-care product or a medical instrument, stopping, and exporting a judgment result;
s6, inquiring a yz _ inststruct table according to the medicine related information identified in the step S5, and deriving a final result, wherein the following three conditions exist:
A. inquiring the medicine and having a corresponding medicine specification;
B. inquiring the medicine without the corresponding medicine specification;
C. the query result is null, indicating that it is not a drug.
8. A photographing and medicine recognizing system based on an OCR recognition technology is characterized by comprising:
the camera module is used for shooting and recording the medicines, the health care products and the medical instruments to be identified;
the OCR recognition module is used for extracting character information of the picture shot by the camera module;
and the keyword matching module is used for matching the character information extracted by the OCR module so as to classify the pictures shot by the camera module into medicines, health-care products, medical instruments or the like.
9. An application method of a photographing and medicine recognizing system based on an OCR recognition technology is characterized by comprising the following steps:
s1, shooting an object to be identified by using a camera module;
s2, extracting character information of the picture shot by the camera module by using an OCR recognition module;
s3, matching the character information extracted by the OCR module by using a keyword matching module so as to classify the object to be recognized: pharmaceuticals, nutraceuticals, medical devices, or the like;
if the output matching result of the object to be identified is a medicine or other objects, inquiring a yz _ inststruct table and deriving a final result;
and if the output matching result of the object to be identified is a health-care product or a medical instrument, stopping outputting the judgment result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211279276.9A CN115620305A (en) | 2022-10-19 | 2022-10-19 | Photographing and medicine recognizing system based on deep learning and using method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211279276.9A CN115620305A (en) | 2022-10-19 | 2022-10-19 | Photographing and medicine recognizing system based on deep learning and using method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115620305A true CN115620305A (en) | 2023-01-17 |
Family
ID=84864870
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211279276.9A Pending CN115620305A (en) | 2022-10-19 | 2022-10-19 | Photographing and medicine recognizing system based on deep learning and using method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115620305A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116168410A (en) * | 2023-04-21 | 2023-05-26 | 江苏羲辕健康科技有限公司 | Medicine box information identification method and system based on neural network |
-
2022
- 2022-10-19 CN CN202211279276.9A patent/CN115620305A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116168410A (en) * | 2023-04-21 | 2023-05-26 | 江苏羲辕健康科技有限公司 | Medicine box information identification method and system based on neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101959831B1 (en) | Apparatus and method for image recognition processing | |
| KR101769918B1 (en) | Recognition device based deep learning for extracting text from images | |
| US20130202159A1 (en) | Apparatus for real-time face recognition | |
| JP2008259156A (en) | Information processing device, information processing system, information processing method, program, and storage medium | |
| US20150169972A1 (en) | Character data generation based on transformed imaged data to identify nutrition-related data or other types of data | |
| Hansen | A Matlab Project in Optical Character Recognition (OCR) | |
| CN108427959A (en) | Board state collection method based on image recognition and system | |
| CN111931548B (en) | Face recognition system, method for establishing face recognition data and face recognition method | |
| CN115620305A (en) | Photographing and medicine recognizing system based on deep learning and using method thereof | |
| Surve et al. | Automatic attendance system using face recognition technique | |
| CN109214302A (en) | One kind being based on multispectral substance identification | |
| JP2700440B2 (en) | Article identification system | |
| Proma et al. | Medicine recognition from colors and text | |
| KR101396713B1 (en) | Apparatus for recognizing medicine using image technique | |
| CN113553892A (en) | Inspection and physical examination report result extraction method based on deep learning and OCR | |
| CN113920335A (en) | Deep learning-based image and text embedded drug label identification method | |
| KR20180126352A (en) | Recognition device based deep learning for extracting text from images | |
| Suntronsuk et al. | Pill image binarization for detecting text imprints | |
| CN110222660B (en) | Signature authentication method and system based on dynamic and static feature fusion | |
| Moetesum et al. | Segmentation and classification of offline hand drawn images for the bgt neuropsychological screening test | |
| Suntronsuk et al. | Automatic text imprint analysis from pill images | |
| JP2989741B2 (en) | Article identification system | |
| KR100421683B1 (en) | Person identifying method using image information | |
| Marciniak et al. | Face recognition from low resolution images | |
| CN112257764A (en) | License plate classification method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |