CN106652673B - Method for automatically identifying and reading drug specification - Google Patents
Method for automatically identifying and reading drug specification Download PDFInfo
- Publication number
- CN106652673B CN106652673B CN201710027181.0A CN201710027181A CN106652673B CN 106652673 B CN106652673 B CN 106652673B CN 201710027181 A CN201710027181 A CN 201710027181A CN 106652673 B CN106652673 B CN 106652673B
- Authority
- CN
- China
- Prior art keywords
- text
- medicine
- voice
- drug
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/001—Teaching or communicating with blind persons
- G09B21/006—Teaching or communicating with blind persons using audible presentation of the information
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Medical Treatment And Welfare Office Work (AREA)
- Medical Preparation Storing Or Oral Administration Devices (AREA)
- Character Input (AREA)
- Character Discrimination (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for automatically identifying and reading a medicine specification. The method comprises the following steps: the system automatically identifies the national drug standard words by shooting a picture of a drug specification, and acquires specific drug information by calling an internet interface; the method comprises the steps of realizing Fourier transform-based rotation text correction through a Hough line detection algorithm, and rotating a text with rotation deviation obtained by shooting to be horizontal; detecting whether the paper is turned over or not by identifying whether characters exist on the medicine specification or not, and turning back the turned paper through voice prompt; acquiring a national drug standard character by simultaneously identifying the picture corrected by the rotating text and the picture obtained after turning 180 degrees; and uploading the Chinese medicine standard characters obtained by identification and returning the medicine information to read aloud by calling a medicine information interface provided by the Internet. The invention can help the elder, the blind, the visual disturbance, the illiterate and other groups read the medicine specification, and the medicine specification is played by voice, thereby improving the life quality of the elder, the blind, the visual disturbance, the illiterate and other groups.
Description
Technical Field
The invention relates to the technical field of identification of medicine specifications, in particular to a method for automatically identifying and reading the medicine specifications.
Background
In China, the number of visually impaired people reaches 1731 ten thousand, and accounts for 18 percent of the total number of blind people in the world. The blind and the amblyopia friends have much inconvenience in life, and the thing that the reading and newspaper reading are easy to be reversed in the view of common people is extremely difficult for the blind and the amblyopia friends. Nevertheless, some braille books and newspapers can be used for learning and reading. However, the drug instruction is generally braille-free, and there is no way to do so when they need to take their medication.
At present, the population of the aged people in China exceeds 2 hundred million, and the aging degree is further deepened. The elderly often deal with medicines in daily life, eyes of the elderly are not good, characters on a medicine specification are small, and the medicine is a headache for the elderly. In China, the number of illiterate groups exceeds 5000 thousands, and how to recognize and read the words on the medicine specification is a difficult thing for the people.
Based on the existing problems, the core function of the invention is to read the medicine specification for the blind and the amblyopia group and to get the medicine specification in the form of voice. Of course, this function is also well applicable to the elderly and illiterate population, as they share common needs. In consideration of the particularity of the user group, the work is operated and controlled through voice, and the work is more efficient and convenient to operate actually. In addition, the invention also adds the function of voice broadcasting the latest social hotspot and opens a window of soul contacting with the outside for blind friends and old people.
Disclosure of Invention
In order to achieve the technical purpose, the invention provides a method for automatically identifying and reading a medicine specification, and the method has the functions of identifying the specification, correcting a rotating text, detecting paper turning, detecting text turning, acquiring medicine information, waking up by voice, synthesizing voice and identifying voice.
A method for automatically identifying and reading a medicine specification adopts an identification system with a camera, a processor and a voice playing module, and comprises the following steps:
(1) the system automatically identifies the national drug standard words by shooting a picture of the drug specification and acquires specific drug information by calling an internet interface;
(2) correcting the rotated text, namely realizing the rotated text correction based on Fourier transform by using a Hough line detection algorithm, and rotating the text with rotational deviation obtained by shooting to be horizontal;
(3) detecting paper turning, namely detecting whether the paper is turned by identifying whether characters exist on the medicine specification, and turning the turned paper back through voice prompt;
(4) text turning detection, namely acquiring Chinese medicine standard characters by simultaneously identifying the pictures corrected by rotating the text and the pictures obtained after turning by 180 degrees;
(5) acquiring medicine information, uploading the recognized national medicine standard characters and returning the medicine information by calling a medicine information interface provided by the Internet;
(6) voice awakening, namely awakening the system from a sleep state by setting an awakening phrase by using a voice awakening library;
(7) and speech synthesis, wherein text information of the medicine is read out by using a speech synthesis library.
Further, the step (1) comprises the following steps:
(1.1) analyzing the connected region, detecting the region outline and the sub-outline of the character region, and integrating the region outline and the sub-outline into a block region;
(1.2) finding a block area, and detecting a character outline to obtain a text line; then obtaining words through spaces;
(1.3) finding text lines and words, and analyzing the words by adopting a self-adaptive classifier; performing word analysis twice;
and (1.4) obtaining a recognition text, recognizing the fuzzy blank space, stroke height and lower case letters.
Further, the step (2) comprises the following steps:
(2.1) reading in an original file in a gray scale mode;
(2.2) expanding the image to a suitable size to facilitate rapid transformation;
(2.3) performing DFT operation, and respectively calculating a real part and an imaginary part;
(2.4) properly adjusting the data, and reducing the range of the numerical value by using a log function in consideration of large amplitude variation range;
(2.5) moving the center, wherein the low-frequency part of the DFT operation result is positioned at four corners, and the high-frequency part is positioned at the center, so that the low-frequency part is moved to the center;
and (2.6) image correction, namely firstly carrying out binarization on the obtained Fourier spectrum, then detecting a straight line, then finding out the oblique line meeting the conditions and obtaining an angle, then carrying out angle conversion, and finally correcting the image.
Further, the step (1) further comprises simplifying an identification word bank, only reserving keywords, and then training to improve identification accuracy and identification speed; and can identify foreign imported drugs. And (4) opening and turning detection, namely detecting whether the paper is turned or not by identifying whether characters exist on the medicine specification, and turning the turned paper back through voice prompt. And text turning detection, namely acquiring the national drug standard characters by simultaneously identifying the pictures corrected by the rotating text and the pictures obtained after turning for 180 degrees, and avoiding that correct results cannot be identified after the text is turned.
Further, the step (5) of obtaining the medicine information further comprises processing the returned character string to obtain the text information of the medicine, and providing a file for voice synthesis.
Furthermore, the method also comprises the step of adopting a voice recognition module to recognize the voice instruction by setting the control command by using a voice recognition library.
After the technical scheme is adopted, the invention at least has the following advantages and technical effects:
(1) the design is novel, and the blind people and the old people are concerned, so the device has social responsibility. The difficulty of reading the medicine specification in daily life is solved practically by giving the reading point to the medicine specification, and the medicine specification reading device has humanistic care.
(2) The Chinese medicine standard character number is used as a keyword for character recognition, so that the problem that local recognition results are inaccurate due to bending deformation of paper is well avoided. Compared with Chinese characters, the recognition accuracy of letters and numbers is higher, and the recognition speed can be further improved by simplifying the character library. The identification method is very ingenious and well solves the problems.
(3) Considering that the blind and the friend can not operate smoothly, the invention adds the functions of paper turning detection, rotary text correction and text turning detection, and the system can not have any obstacle when operating.
(4) The invention fully considers the particularity of the user group, is operated by voice control, is very convenient and efficient in actual operation and has wide application prospect.
(5) The invention also has a voice awakening function, when the system is needed, the system can be awakened by only lightly speaking the 'my small assistant', and when the system is not needed, the system can be quitted by speaking the 'quit', so that the system works at low power consumption.
(6) The invention also has the function of broadcasting the social hotspots, well provides news information for blind friends and old people, and enriches the daily life of the blind friends and the old people.
Drawings
FIG. 1 is a block diagram of the steps of a method of the present invention for automatically identifying and reading drug instructions.
FIG. 2 is a block diagram of the steps for automatically identifying and reading a pharmaceutical product description according to the method of the present invention.
Detailed Description
The practice of the present invention will be further illustrated, but is not limited, by the following figures and examples.
As shown in fig. 1, a method for automatically recognizing and reading a drug instruction in this embodiment employs a recognition system having a camera, a processor, and a voice playing module, and includes the following steps:
(1) the system automatically identifies the national drug standard words by shooting a picture of the drug specification and acquires specific drug information by calling an internet interface;
(2) correcting the rotated text, namely realizing the rotated text correction based on Fourier transform by using a Hough line detection algorithm, and rotating the text with rotational deviation obtained by shooting to be horizontal;
(3) detecting paper turning, namely detecting whether the paper is turned by identifying whether characters exist on the medicine specification, and turning the turned paper back through voice prompt;
(4) text turning detection, namely acquiring Chinese medicine standard characters by simultaneously identifying the pictures corrected by rotating the text and the pictures obtained after turning by 180 degrees;
(5) acquiring medicine information, uploading the recognized national medicine standard characters and returning the medicine information by calling a medicine information interface provided by the Internet;
(6) voice awakening, namely awakening the system from a sleep state by setting an awakening phrase by using a voice awakening library;
(7) and speech synthesis, wherein text information of the medicine is read out by using a speech synthesis library.
By way of example, a test is carried out with a photograph of a text rotated by 15 °, which shows that the picture has been rotated to a horizontal position after the text has been rotated, and the accuracy of the subsequent recognition instruction can be well ensured.
As shown in fig. 2, step (1) includes the following steps:
(1.1) analyzing the connected region, detecting the region outline and the sub-outline of the character region, and integrating the region outline and the sub-outline into a block region;
(1.2) finding a block area, and detecting a character outline to obtain a text line; then obtaining words through spaces;
(1.3) finding text lines and words, and analyzing the words by adopting a self-adaptive classifier; performing word analysis twice;
and (1.4) obtaining a recognition text, recognizing the fuzzy blank space, stroke height and lower case letters.
The step (2) comprises the following steps:
(2.1) reading in an original file in a gray scale mode;
(2.2) expanding the image to a suitable size to facilitate rapid transformation;
(2.3) performing DFT operation, and respectively calculating a real part and an imaginary part;
(2.4) properly adjusting the data, and reducing the range of the numerical value by using a log function in consideration of large amplitude variation range;
(2.5) moving the center, wherein the low-frequency part of the DFT operation result is positioned at four corners, and the high-frequency part is positioned at the center, so that the low-frequency part is moved to the center;
and (2.6) image correction, namely firstly carrying out binarization on the obtained Fourier spectrum, then detecting a straight line, then finding out the oblique line meeting the conditions and obtaining an angle, then carrying out angle conversion, and finally correcting the image.
The step (1) also comprises the steps of simplifying an identification word stock, only reserving keywords, and then training to improve the identification accuracy and the identification speed; and can identify foreign imported drugs. And (4) opening and turning detection, namely detecting whether the paper is turned or not by identifying whether characters exist on the medicine specification, and turning the turned paper back through voice prompt. And text turning detection, namely acquiring the national drug standard characters by simultaneously identifying the pictures corrected by the rotating text and the pictures obtained after turning for 180 degrees, and avoiding that correct results cannot be identified after the text is turned.
As an example, to facilitate taking a picture, the original picture text taken is not horizontally placed. For correct recognition, the photograph needs to be rotated 90 ° clockwise before recognition can be performed. As the product only needs to identify the serial number of the national standard characters, the Chinese standard characters of production enterprises at the modification period of the term execution standard approval document number specification are only added into the word stock, wherein the Arabic numerals ' 0-9 ', the capital letters ' H, Z, S, B, T, F, J ', and the Chinese characters ' the main treatment method of the drug name, the component and the character, the adverse reaction and the contraindication of the adverse reaction of the main treatment method and the attention matters are added into the mutual action storage package. Although the identified information appears to have no logic, doing so has the following two benefits. The method has the advantages that: the word stock is small, and the recognition speed is high; the advantages are two: and the keywords are limited, namely one layer of filtering is performed, so that the identification accuracy is improved. The word stock obtained by the training of the method is used for recognition, the recognition time for recognizing one instruction book is only about 5s, the recognition time for recognizing the default Chinese word stock is about 1min, and the recognition speed is greatly improved. Moreover, as long as the serial numbers of the matched national medicine standard characters are identified, the result shows that the standard characters Z44022935 are well identified, so that the following operations can call the cloud interface to acquire the detailed information of the medicine through the Z44022935 parameter.
And returning the medicine information of the modified Huoxiangzhengqi pill by calling a medicine information interface provided by the Internet. It can be seen that the original information returned is a JSON string and contains many symbols, so the text can only be read after processing. Regular expressions are used here to process the original information into text for viewing and for the speech synthesis program, respectively. It can be seen that the processing results are good and the processing speed is very fast.
Claims (3)
1. A method for automatically identifying and reading a medicine specification adopts an identification system with a camera, a processor and a voice playing module, and is characterized by comprising the following steps:
(1) the system automatically identifies the national drug standard words by shooting a picture of the drug specification and acquires specific drug information by calling an internet interface; the method also comprises a simplified recognition word library, only the keywords are reserved, and then training is carried out, so that the recognition accuracy and the recognition speed are improved; meanwhile, imported drugs can be identified; the method comprises the following steps:
analyzing the connected region, detecting the region outline and the sub-outline of the character region, and integrating the region outline and the sub-outline into a block region;
finding a block area, and detecting a character outline to obtain a text line; then obtaining words through spaces;
finding text lines and words, and analyzing the words by adopting a self-adaptive classifier; performing word analysis twice;
obtaining an identification text, and identifying a blank containing fuzzy, stroke height and lower case letters;
(2) correcting the rotated text, namely realizing the rotated text correction based on Fourier transform by using a Hough line detection algorithm, and rotating the text with rotational deviation obtained by shooting to be horizontal; the method comprises the following steps:
(2.1) reading in an original file in a gray scale mode;
(2.2) expanding the image to a suitable size to facilitate rapid transformation;
(2.3) performing DFT operation, and respectively calculating a real part and an imaginary part;
(2.4) properly adjusting the data, and reducing the range of the numerical value by using a log function in consideration of large amplitude variation range;
(2.5) moving the center, wherein the low-frequency part of the DFT operation result is positioned at four corners, and the high-frequency part is positioned at the center, so that the low-frequency part is moved to the center;
(2.6) image correction, namely firstly carrying out binarization on the obtained Fourier spectrum, then detecting straight lines, then finding out the oblique lines meeting the conditions and obtaining angles, then carrying out angle conversion, and finally correcting the image;
(3) detecting paper turning, namely detecting whether the paper is turned by identifying whether characters exist on the medicine specification, and turning the turned paper back through voice prompt;
(4) detecting inversion and turnover of the text, and acquiring the national drug standard characters by simultaneously identifying the picture corrected by rotating the text and the picture obtained after turnover by 180 degrees;
(5) acquiring medicine information, uploading the recognized national medicine standard characters and returning the medicine information by calling a medicine information interface provided by the Internet;
(6) voice awakening, namely awakening the system from a sleep state by setting an awakening phrase by using a voice awakening library;
(7) and speech synthesis, wherein text information of the medicine is read out by using a speech synthesis library.
2. The method for automatically recognizing and reading the drug instruction book of claim 1, wherein the step (5) of obtaining the drug information further comprises processing the returned character string to obtain the text information of the drug, and providing a file for speech synthesis.
3. The method of claim 1, further comprising recognizing the voice command using a voice recognition module by setting the control command using a voice recognition library.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710027181.0A CN106652673B (en) | 2017-01-16 | 2017-01-16 | Method for automatically identifying and reading drug specification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710027181.0A CN106652673B (en) | 2017-01-16 | 2017-01-16 | Method for automatically identifying and reading drug specification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106652673A CN106652673A (en) | 2017-05-10 |
| CN106652673B true CN106652673B (en) | 2020-09-22 |
Family
ID=58844304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710027181.0A Active CN106652673B (en) | 2017-01-16 | 2017-01-16 | Method for automatically identifying and reading drug specification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106652673B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107463930A (en) * | 2017-08-11 | 2017-12-12 | 哈尔滨工业大学 | A kind of chip component angle acquisition methods based on frequency domain character |
| CN109919155B (en) * | 2019-03-13 | 2021-03-12 | 厦门商集网络科技有限责任公司 | Inclination angle correction method for text image and terminal |
| CN116168376B (en) * | 2023-04-18 | 2023-07-18 | 苏州大学 | Voice broadcast medicine box recognition device and medicine box recognition method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101149790A (en) * | 2007-11-14 | 2008-03-26 | 哈尔滨工程大学 | Chinese printing style formula identification method |
| CN101493996A (en) * | 2009-01-15 | 2009-07-29 | 北方工业大学 | Intelligent reader and implementation method thereof |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102376304B (en) * | 2010-08-10 | 2014-04-30 | 鸿富锦精密工业（深圳）有限公司 | Text reading system and text reading method thereof |
| CN105550643A (en) * | 2015-12-08 | 2016-05-04 | 小米科技有限责任公司 | Medical term recognition method and device |
| CN105956588A (en) * | 2016-04-21 | 2016-09-21 | 深圳前海勇艺达机器人有限公司 | Method of intelligent scanning and text reading and robot device |
-
2017
- 2017-01-16 CN CN201710027181.0A patent/CN106652673B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101149790A (en) * | 2007-11-14 | 2008-03-26 | 哈尔滨工程大学 | Chinese printing style formula identification method |
| CN101493996A (en) * | 2009-01-15 | 2009-07-29 | 北方工业大学 | Intelligent reader and implementation method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106652673A (en) | 2017-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Joshi et al. | Efficient multi-object detection and smart navigation using artificial intelligence for visually impaired people | |
| CN106575500B (en) | Method and apparatus for synthesizing speech based on facial structure | |
| CN106652673B (en) | Method for automatically identifying and reading drug specification | |
| CN103838866B (en) | A kind of text conversion method and device | |
| US11263403B2 (en) | Interpreting a most likely meaning of a phrase | |
| US11176126B2 (en) | Generating a reliable response to a query | |
| US20200004738A1 (en) | Generating further knowledge to process query | |
| Hagargund et al. | Image to speech conversion for visually impaired | |
| Qin et al. | Finger-vein verification based on LSTM recurrent neural networks | |
| CN109871440B (en) | Intelligent prompting method, device and equipment based on semantic analysis | |
| Kutzner et al. | Writer identification using handwritten cursive texts and single character words | |
| Rivera-Acosta et al. | Spelling correction real-time american sign language alphabet translation system based on yolo network and LSTM | |
| Padmanabhan et al. | Doctors Handwritten Prescription Recognition System In Multi Language Using Deep Learning | |
| US20200175064A1 (en) | Image processing utilizing an entigen construct | |
| US10133920B2 (en) | OCR through voice recognition | |
| Berkol et al. | Visual Lip Reading Dataset in Turkish | |
| US20220100744A1 (en) | Generating a timely response to a query | |
| Ravi et al. | Raspberry pi based smart reader for blind people | |
| De Jonge et al. | Learning on the Field: L2 Turkish Vowel Production by L1 American English-Speaking NGOs in Turkey | |
| Onuean et al. | Burapha-TH: A multi-purpose character, digit, and syllable handwriting dataset | |
| Zhang et al. | LFLDNet: Lightweight Fingerprint Liveness Detection Based on ResNet and Transformer | |
| Majid et al. | Digitization of Handwritten Chess Scoresheets with a BiLSTM Network | |
| Vashisth et al. | Hand Gesture Recognition in Indian Sign Language Using Deep Learning | |
| Petinou et al. | Promoting speech intelligibility through phonologically dense targets | |
| Kumar et al. | Translation of Multilingual Text into Speech for Visually Impaired Person |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |