CN114550174A - Reading assisting method and device - Google Patents
Reading assisting method and device Download PDFInfo
- Publication number
- CN114550174A CN114550174A CN202210116492.5A CN202210116492A CN114550174A CN 114550174 A CN114550174 A CN 114550174A CN 202210116492 A CN202210116492 A CN 202210116492A CN 114550174 A CN114550174 A CN 114550174A
- Authority
- CN
- China
- Prior art keywords
- read
- scanning device
- scanning
- character image
- reading
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000001514 detection method Methods 0.000 claims description 31
- 230000007246 mechanism Effects 0.000 claims description 24
- 230000009471 action Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 13
- 230000001915 proofreading effect Effects 0.000 claims description 8
- 238000012937 correction Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 description 9
- 208000029257 vision disease Diseases 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 6
- 238000003786 synthesis reaction Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000003062 neural network model Methods 0.000 description 4
- 238000012015 optical character recognition Methods 0.000 description 4
- 206010047571 Visual impairment Diseases 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000004393 visual impairment Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000002950 deficient Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241001672694 Citrus reticulata Species 0.000 description 1
- 206010021703 Indifference Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Character Input (AREA)
Abstract
The invention provides an auxiliary reading method and an auxiliary reading device, wherein the auxiliary reading method comprises the following steps: acquiring a character image to be read in a scanning area based on a scanning device; determining text information to be read based on the character image to be read; outputting voice data to be read based on the text information to be read; and sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device. According to the reading assisting method and the reading assisting device, the text image to be read is converted into the voice data to be read, the scanning area is switched through the scanning device, and the automatic page turning function is achieved, so that the vision-impaired people can be assisted to read smoothly, the reading assisting flexibility is improved, and the real-time reading requirements of people can be met.
Description
Technical Field
The invention relates to the technical field of computers, in particular to an auxiliary reading method and device.
Background
For the group with vision disorder, it is difficult to read the text content normally in daily life, so reading the medicine specification while taking medicine, and reading the paper book or electronic book has a great obstacle, and it is difficult to finish the reading action independently, and how to help the vision disorder to assist reading becomes a technical problem worthy of study.
The existing auxiliary reading method mainly prepares voice data corresponding to text information in advance, and reads the corresponding voice data from a local or network terminal when auxiliary reading is needed.
Disclosure of Invention
The invention provides an auxiliary reading method and device, which are used for solving the defects that resources are solidified, functions are single, auxiliary reading cannot be flexibly carried out, and real-time reading requirements of people are difficult to meet in the prior art, and realizing an automatic page turning function, so that people with visual impairment can be assisted to smoothly read, the flexibility of auxiliary reading is improved, and the real-time reading requirements of people can be met.
The invention provides an auxiliary reading method, which comprises the following steps: acquiring a character image to be read in a scanning area based on a scanning device; determining text information to be read based on the character image to be read; outputting voice data to be read based on the text information to be read; and sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device.
According to the reading assisting method provided by the invention, the acquisition of the character image to be read in the scanning area based on the scanning device comprises the following steps: acquiring a character image to be detected in the scanning area based on the scanning device; carrying out boundary detection on the character image to be detected to obtain a boundary detection result; and taking the character image to be detected as the character image to be read under the condition that the boundary detection result is aligned.
According to the reading assisting method provided by the invention, after the boundary detection is performed on the character image to be detected to obtain a boundary detection result, the reading assisting method further comprises the following steps: when the boundary detection result is a deviation, outputting position adjustment prompt information or sending a position adjustment instruction to the scanning device so that the scanning device can execute an adjustment action to adjust the position of the scanning area; and the position adjustment prompt information is used for prompting a user to manually adjust the position of the scanning area.
According to the reading assisting method provided by the invention, the scanning device comprises: the page number fixing mechanism and the mechanical arm are used for sending a scanning area switching instruction to the scanning device, and the page number fixing mechanism comprises: under the condition that a medium to be scanned is a paper book, sending a force adjusting instruction to the page fixing mechanism so that the page fixing mechanism releases page turning limit of the paper book; and sending a page turning control instruction to the mechanical arm so that the mechanical arm can pick up and turn pages of the paper book to switch the page number of the paper book. According to the reading assisting method provided by the invention, the scanning device comprises: the mechanical arm sends a scanning area switching instruction to the scanning device, and the scanning area switching instruction comprises the following steps: and under the condition that the medium to be scanned is the electronic book, sending a sliding control instruction to the mechanical arm so that the mechanical arm can slide and turn pages of the electronic book to switch the page number of the electronic book. According to the auxiliary reading method provided by the invention, the scanning device further comprises an identifier; the sending of the scan area switching instruction to the scanning device previously further includes: controlling the identifier to identify the type of the medium to be scanned, wherein the type of the medium to be scanned comprises: a paper book or an electronic book.
According to the reading assisting method provided by the invention, the updating of the character image to be read based on the scanning device comprises the following steps: acquiring a character image to be corrected in the scanning area based on the scanning device; performing page number proofreading on the character image to be proofread to obtain a page number proofreading result; and under the condition that the page number correction result is correct, updating the character image to be read based on the character image to be corrected.
The invention also provides an auxiliary reading device, comprising: the acquisition module is used for acquiring character images to be read in the scanning area based on the scanning device; the determining module is used for determining text information to be read based on the character image to be read; the output module is used for outputting the voice data to be read based on the text information to be read; and the sending module is used for sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of any one of the reading assisting methods.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the reading aid method as described in any of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the reading aid method as described in any one of the above.
According to the reading assisting method and the reading assisting device, the text image to be read is converted into the voice data to be read, the scanning area is switched through the scanning device, and the automatic page turning function is achieved, so that the vision-impaired people can be assisted to read smoothly, the reading assisting flexibility is improved, and the real-time reading requirements of people can be met.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart of an assisted reading method according to the present invention;
FIG. 2 is a schematic view of an auxiliary reading device according to the present invention;
fig. 3 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The reading assistance method and apparatus of the present invention are described below with reference to fig. 1 to 3.
It can be understood that the auxiliary reading method can be executed by an auxiliary reading device, where the auxiliary reading device can be an electronic device, the electronic device can include a processing device such as a mobile phone, a tablet computer, a notebook computer, or a desktop computer, and an execution mechanism, the execution mechanism can be electrically connected to the processing device through a data interface, and can also be wirelessly connected to the execution mechanism, of course, the electronic device can also be an integrated robot, and the robot can help a user to perform auxiliary reading.
As shown in fig. 1, the present invention provides an assistant reading method, which includes the following steps 110 to 140.
And 110, acquiring a character image to be read in the scanning area based on the scanning device.
It can be understood that the medium to be scanned may be placed on the auxiliary reading device, the medium to be scanned may be an object having text information, for example, a paper medium on which text information is recorded, such as a paper book, a newspaper, a single sheet of a specification, or a single sheet of a poster, and may also be a label attached to various articles, on which text information is recorded, or an electronic book or other display screen on which text information is recorded, which is displayed on a display screen of an electronic device.
This supplementary reading device can have scanning device, and scanning device can have the camera, and the camera can be one or more, and the camera lens of camera is towards the scanning area, and the camera can be shot the characters image of waiting to read in the scanning area.
The character image to be read is data in a picture form, the auxiliary reading device can also be provided with a processor, the scanning device is in communication connection with the processor, and the scanning device can send the character image to be read to the processor of the auxiliary reading device after acquiring the character image to be read.
And step 120, determining text information to be read based on the character image to be read.
It can be understood that after the text image to be read is acquired, the processor may perform text Recognition on the text image to be read, extract text content from the text image to be read, and for example, may combine an OCR (Optical Character Recognition) technology with an ICR (Intelligent Character Recognition) technology to realize accurate Recognition of the text image to be read.
OCR refers to a process in which an electronic device such as a scanner or a digital camera examines a character printed or written on a medium, determines its shape by a certain detection mode, and then translates the shape into a computer word by a character recognition method.
ICR is an artificial intelligence technology of computer deep learning implanted on the basis of OCR so as to improve the indexes of the error recognition rate, the recognition speed and the like.
It is to be noted that, in the process of determining the text information to be read from the text image to be read, the neural network model may be implemented, and the neural network model may be trained, for example, the neural network model may be trained in a supervised learning manner or an unsupervised learning manner, so as to improve the recognition accuracy of the neural network model.
The specific method for determining the text information to be read according to the text image to be read is not limited herein, and those skilled in the art can select a feasible method according to the specific situation to implement.
And step 130, outputting the voice data to be read based on the text information to be read.
It can be understood that after the Text information To be read is obtained, the Text information To be read may be converted into Speech data To be read, that is, Text content available for reading may be converted into human voice data that can be played, for example, TTS (Text To Speech ) technology may be used To convert the Text information To be read into Speech data To be read.
The TTS technology can intelligently convert characters into natural voice streams through the design of a neural network under the support of a built-in chip. The TTS technology carries out real-time conversion on the text file, and the conversion time can be calculated in seconds. Under the action of the special intelligent voice controller, the voice rhythm of the text output is smooth, so that a listener feels natural when listening to information and does not have the indifference and acerbity feeling of machine voice output.
The TTS speech synthesis technology can cover national standard first-level and second-level Chinese characters, has an English interface, automatically identifies Chinese and English, and supports mixed reading of Chinese and English. All the sounds adopt the real Mandarin as the standard pronunciation, the rapid speech synthesis of 120-150 Chinese characters/minute is realized, the reading speed reaches 3-4 Chinese characters/second, and the user can hear clear and pleasant tone quality and coherent and smooth intonation.
TTS is one type of speech synthesis application that converts files stored in a computer, such as help files or web pages, into natural speech output. TTS can not only help visually impaired people read information on a computer, but also increase the readability of text documents. TTS applications include voice-driven mail and voice-sensitive systems, and are often used with voice recognition programs.
Converting the text information to be read into the voice data to be read may include at least three processes: first, text analysis, linguistic analysis of the input text, lexical, grammatical and semantic analysis, sentence by sentence, to determine the low-level structure of the sentence and the composition of the phonemes of each word, including text break, word segmentation, processing of polyphones, processing of numbers, processing of abbreviations, etc. Secondly, voice synthesis, extracting the single characters or phrases corresponding to the processed text from a voice synthesis library, and converting the linguistic description into a speech waveform. Third, prosody generation refers to the quality of speech output by a speech synthesis system, and is generally subjective in terms of intelligibility, naturalness, and coherence. Clarity is the percentage of meaningful words that are correctly heard; the naturalness is used for evaluating whether the tone quality of the synthesized voice is close to the voice of a person and whether the tone of the synthesized word is natural; coherence is used to evaluate whether a synthesized sentence is fluent.
The method for obtaining the voice data to be read according to the text information to be read is not limited herein, and a person skilled in the art can select the method according to the actual situation.
It can be understood that the auxiliary reading device can have an individual reading mode and a continuous reading mode, the individual reading mode refers to only reading the scanning area once, that is, after the scanning area is determined, the text image to be read in the scanning area is converted into the text information to be read only once, the text information to be read is converted into the voice data to be read, the voice data to be read is played, and after the voice data to be read is played, the auxiliary reading process is completed.
The continuous reading mode refers to that multiple scanning areas are used for reading for multiple times, the scanning areas can be switched, for example, when the scanning areas are pages on a paper book, the paper book can be continuously read, and after auxiliary reading of one page is completed, auxiliary reading of the next page can be performed.
When the continuous reading mode is started, after the text information to be read obtained by scanning at this time is converted into voice data to be read and played, a scanning area switching instruction can be sent to the scanning device, so that the scanning device is controlled to execute target action.
The scanning device may include a camera and a mechanical arm, the target action may be to move a lens of the camera toward to move a position of the scanning area, or to turn pages of a paper book by the mechanical arm, or to slide and touch an electronic book by the mechanical arm to turn pages, where a specific action form is not limited here.
It is worth noting that, in the current method for reading in an auxiliary manner, voice data corresponding to text information is prepared in advance, and the corresponding voice data is read from a local or network side when auxiliary reading is needed.
Through the design of this kind of automatic switch-over scanning area of this embodiment, just can realize treating the position switch of scanning medium, can realize treating the continuous reading of scanning medium, can realize the automatic page turning to books, people just need not manual book turning when using, can greatly facilitate the user and develop the reading, especially can play great supplementary reading effect to the crowd of visual impairment for the crowd of visual impairment can smoothly acquire the knowledge on the books, can master the text content on treating the scanning medium conveniently.
According to the auxiliary reading method provided by the invention, the text image to be read is converted into the voice data to be read, and the scanning area is switched through the scanning device, so that the automatic page turning function is realized, the visual disorder people can be assisted to smoothly read, the auxiliary reading flexibility is improved, and the real-time reading requirement of people can be met.
In some embodiments, the step 110 of acquiring the text image to be read in the scanning area based on the scanning device includes: acquiring a character image to be detected in a scanning area based on a scanning device; carrying out boundary detection on the character image to be detected to obtain a boundary detection result; and under the condition that the boundary detection result is alignment, taking the character image to be detected as the character image to be read.
It can be understood that after the character image to be detected is acquired by the scanning device, the boundary detection may be performed on the character image to be detected, for example, the character features in the character image to be detected may be detected, whether the character features are defective or not is determined, if the character features are defective, the boundary of the character image to be detected is considered to be misaligned, the boundary region of the character image to be detected may also be detected to be compared with a preset boundary template image, whether the boundary of the character image to be detected is aligned or not is determined, and a specific boundary detection method is not limited here.
If the boundary detection result of the character image to be detected is judged to be aligned, the character image to be detected can be directly used as the character image to be read, a subsequent character recognition process is carried out, boundary detection is carried out at the position, interference caused by boundary deviation can be reduced, and accuracy of subsequent character recognition can be improved.
In some embodiments, after performing boundary detection on the text image to be detected and obtaining a boundary detection result, the method further includes: when the boundary detection result is a deviation, outputting position adjustment prompt information or sending a position adjustment instruction to the scanning device so that the scanning device can execute an adjustment action to adjust the position of the scanning area; the position adjustment prompt information is used for prompting a user to manually adjust the position of the scanning area.
It can be understood that when the boundary detection result obtained by performing boundary detection on the character image to be detected is a deviation, it indicates that the character image to be detected has a deviation in boundary, at this time, the processor may output an adjustment prompt message, for example, the adjustment prompt message may be played through a speaker, and after hearing the adjustment prompt message, the user may manually adjust the position of the scanning area, thereby correcting the problem of the deviation in boundary of the character image to be detected.
Of course, the processor may also send a position adjustment instruction to the scanning device, and after receiving the position adjustment instruction, the scanning device may perform an adjustment action to adjust the position of the scanning area, for example, the position of the scanning area may be adjusted by the mechanical arm, so as to correct the problem of the deviation degree of the boundary of the text image to be detected.
In some embodiments, the step 120 of determining the text information to be read based on the text image to be read includes: and inputting the character image to be read into the text recognition model, and acquiring the text information to be read output by the text recognition model. The text recognition model is obtained by training by taking a text sample image to be read as a sample and taking text information sample data to be read corresponding to the text sample image to be read as a sample label.
It can be understood that the text recognition model may be a deep learning neural network, such as a convolutional neural network or a residual neural network, the text recognition model may be obtained by training a to-be-read text image data set including a large number of training samples, after the text recognition model is trained, accurate recognition and detection of the to-be-read text image may be realized, and text information to be read may be output by inputting the to-be-read text image into the text recognition model.
The deep learning neural network used by the text recognition model can pick out the features in the input text sample image to be read, each feature is used for obtaining an output result, each output result is compared with a sample label, the sample label is the text information sample data to be read corresponding to the text sample image to be read, the features meeting the requirements through comparison can be reserved, the features meeting the requirements through comparison are ignored through Loss parameters, the core features needing to be memorized can be learned finally through continuous iterative training of a large number of input text sample images to be read, the different core features can be classified, and finally the newly input text image to be read can be distinguished according to the core features.
Before the text recognition model is trained, the filter of the convolution layer of the deep learning neural network is completely random, the filter is not activated for any feature, namely, any feature cannot be detected, in the training process, the weight of the blank filter is modified so that the blank filter can detect a specific scene, which is a supervised learning mode, and based on the supervised learning mode, the deep learning neural network can learn the core features required by self so as to judge the newly input character image to be read according to the core features.
The text recognition model is obtained by training a text image data set to be read, the text image data set to be read can comprise various text sample images to be read, and text information sample data to be read corresponding to the text sample images to be read can be artificially given and can be correct text content in the text sample images to be read.
In some embodiments, the scanning device comprises an identifier, a page fixing mechanism and a mechanical arm, and the sending of the scan area switching instruction to the scanning device comprises: identifying the type of the medium to be scanned through an identifier; under the condition that the medium to be scanned is a paper book, sending a force adjusting instruction to the page fixing mechanism so that the page fixing mechanism releases page turning limit of the paper book; and sending a page turning control instruction to the mechanical arm so that the mechanical arm can pick up and turn pages of the paper books to switch page numbers of the paper books.
It will be appreciated that this function is used when a continuous reading mode is performed, such as a book that requires multiple pages to be read in an assisted manner. The process of the continuous reading mode can realize automatic page turning of the book. The scanning device of the auxiliary reading device can comprise a matched book scanning support, when text content on a book with paper or other media is converted into voice to be output, the medium to be scanned is placed on the support in a scanning area, auxiliary reading can automatically identify whether the medium to be scanned is a paper book or an electronic book on the bottom according to the characteristics of the medium to be scanned or the using mode of the support, for example, according to the difference of the shape, the material and the like of the medium to be scanned, and the auxiliary reading device is combined with the using condition of the book scanning support, for example, the paper book needs to be fixed by page fixing mechanisms from the left direction and the right direction, and the electronic book only needs to be fixed with electronic equipment, so that the medium to be scanned is the paper book or the electronic book automatically. And according to different scanning object media, different instructions are sent to the mechanical arm, and a corresponding page turning method is called.
For example, for a paper book, after it is confirmed that the text-to-speech operation on the current scanning area is completed, the program may be run: the page number fixing mechanism on the right side is properly loosened and slowly deviated, and meanwhile, the mechanical fingers of the mechanical arm are called to simulate the book turning action of a human, so that pages are turned by one page and placed under the page number fixing mechanism on the left side.
In some embodiments, after identifying the type of the medium to be scanned by the identifier, the method further includes: and under the condition that the medium to be scanned is the electronic book, sending a sliding control instruction to the mechanical arm so that the mechanical arm slides and turns pages of the electronic book to switch the page number of the electronic book.
For an electronic book, a page number fixing mechanism is not needed, a mechanical finger of a mechanical arm is directly called to simulate the page turning action of a person on an electronic product, and the mechanical finger made of a material capable of touching a screen is used for making a screen sliding action upwards or rightwards, so that the page turning is realized. After page turning, the method is similar to that of a paper book, page number detection and correction and error correction are firstly carried out, and if the page number is correct, a new round of image scanning, text recognition and voice reading is carried out.
In some embodiments, updating the text image to be read based on the scanning device includes: acquiring a character image to be corrected in a scanning area based on a scanning device; performing page number proofreading on the character image to be proofread to obtain a page number proofreading result; and under the condition that the page number correction result is correct, updating the character image to be read based on the character image to be corrected.
It can be understood that after the switching of the scanning area is completed, that is, the pages of the medium to be scanned are turned, a new page number may be automatically detected, for example, the common positions of the page numbers of the paper books in the left lower part, the right lower part, the left upper part, the right upper part, the left middle part or the right middle part of the whole scanning area may be checked, and whether the page turning is correct or not may be checked, and if the page turning is correct, a new round of the program "image scanning-text recognition-voice reading" is entered; if not, judging according to the current page number and the page number which has finished reading recently, continuing to turn to the correct continuous page number according to the judgment result, and then entering a new round of 'image scanning-text recognition-voice reading' program.
The following describes the reading aid provided by the present invention, and the reading aid described below and the reading aid described above can be referred to correspondingly.
As shown in fig. 2, the present invention further provides an auxiliary reading device, including: an acquisition module 210, a determination module 220, an output module 230, and a transmission module 240.
And the acquisition module 210 is configured to acquire the text image to be read in the scanning area based on the scanning device.
The determining module 220 is configured to determine text information to be read based on the text image to be read.
And an output module 230, configured to output the voice data to be read based on the text information to be read.
The sending module 240 is configured to send a scan area switching instruction to the scanning device, so that the scanning device switches the scan area and updates the text image to be read based on the scanning device.
The auxiliary reading method can be executed by an auxiliary reading device, the auxiliary reading device can be an electronic device, the electronic device can include a processing device such as a mobile phone, a tablet computer, a notebook computer or a desktop computer and an execution mechanism, the execution mechanism can be electrically connected with the processing device through a data interface and can also be in wireless communication connection with the execution mechanism, of course, the electronic device can also be an integrated robot, and the robot can help a user to perform auxiliary reading.
The reading aid may include a processor that may have an acquisition module 210, a determination module 220, an output module 230, and a transmission module 240, and the acquisition module 210, the determination module 220, the output module 230, and the transmission module 240 may be electrically connected in sequence for performing the reading aid method in sequence.
In some embodiments, the acquisition module is further to: acquiring a character image to be detected in a scanning area based on a scanning device; carrying out boundary detection on the character image to be detected to obtain a boundary detection result; and under the condition that the boundary detection result is alignment, taking the character image to be detected as the character image to be read.
In some embodiments, the acquisition module is further to: when the boundary detection result is a deviation, outputting position adjustment prompt information or sending a position adjustment instruction to the scanning device so that the scanning device can execute an adjustment action to adjust the position of the scanning area; the position adjustment prompt information is used for prompting a user to manually adjust the position of the scanning area.
In some embodiments, the determining module is further to: inputting the character image to be read into a text recognition model, and acquiring text information to be read output by the text recognition model; the text recognition model is obtained by training by taking a text sample image to be read as a sample and taking text information sample data to be read corresponding to the text sample image to be read as a sample label.
In some embodiments, the scanning device comprises an identifier, a page fixing mechanism, and a robotic arm, and in some embodiments, the sending module is further configured to: identifying the type of a medium to be scanned through an identifier; under the condition that the medium to be scanned is a paper book, sending a force adjusting instruction to the page fixing mechanism so that the page fixing mechanism releases page turning limit of the paper book; and sending a page turning control instruction to the mechanical arm so that the mechanical arm can pick up and turn pages of the paper books to switch the page numbers of the paper books.
In some embodiments, the sending module is further configured to: and under the condition that the medium to be scanned is the electronic book, sending a sliding control instruction to the mechanical arm so that the mechanical arm slides and turns pages of the electronic book to switch the page number of the electronic book.
In some embodiments, the sending module is further configured to: acquiring a character image to be corrected in a scanning area based on a scanning device; performing page number proofreading on the character image to be proofread to obtain a page number proofreading result; and under the condition that the page number correction result is correct, updating the character image to be read based on the character image to be corrected.
According to the reading assisting method and the reading assisting device, the text image to be read is converted into the voice data to be read, the scanning area is switched through the scanning device, and the automatic page turning function is achieved, so that the vision-impaired people can be assisted to read smoothly, the reading assisting flexibility is improved, and the real-time reading requirements of people can be met.
Fig. 3 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 3: a processor (processor)310, a communication Interface (communication Interface)320, a memory (memory)330 and a communication bus 340, wherein the processor 310, the communication Interface 320 and the memory 330 communicate with each other via the communication bus 340. The processor 310 may invoke logic instructions in the memory 330 to perform a read-assist method comprising: acquiring a character image to be read in a scanning area based on a scanning device; determining text information to be read based on the character image to be read; outputting voice data to be read based on the text information to be read; and sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device.
According to the auxiliary reading method provided by the invention, the text image to be read is converted into the voice data to be read, and the scanning area is switched through the scanning device, so that the automatic page turning function is realized, the visual disorder people can be assisted to smoothly read, the auxiliary reading flexibility is improved, and the real-time reading requirement of people can be met.
In addition, the logic instructions in the memory 330 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, a computer is capable of executing the reading assistance method provided by the above methods, and the method includes: acquiring a character image to be read in a scanning area based on a scanning device; determining text information to be read based on the character image to be read; outputting voice data to be read based on the text information to be read; and sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device.
According to the auxiliary reading method provided by the invention, the text image to be read is converted into the voice data to be read, and the scanning area is switched through the scanning device, so that the automatic page turning function is realized, the visual disorder people can be assisted to smoothly read, the auxiliary reading flexibility is improved, and the real-time reading requirement of people can be met.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the reading assistance method provided by the above methods, the method comprising: acquiring a character image to be read in a scanning area based on a scanning device; determining text information to be read based on the character image to be read; outputting voice data to be read based on the text information to be read; and sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device.
According to the auxiliary reading method provided by the invention, the text image to be read is converted into the voice data to be read, and the scanning area is switched through the scanning device, so that the automatic page turning function is realized, the visual disorder people can be assisted to smoothly read, the auxiliary reading flexibility is improved, and the real-time reading requirement of people can be met.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An assistive reading method, comprising:
acquiring a character image to be read in a scanning area based on a scanning device;
determining text information to be read based on the character image to be read;
outputting voice data to be read based on the text information to be read;
and sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device.
2. The reading aid method according to claim 1, wherein the scanning-based device collects text images to be read in a scanning area, and the method comprises:
acquiring a character image to be detected in the scanning area based on the scanning device;
carrying out boundary detection on the character image to be detected to obtain a boundary detection result;
and taking the character image to be detected as the character image to be read under the condition that the boundary detection result is aligned.
3. The reading aid method according to claim 2, wherein after the boundary detection is performed on the text image to be detected to obtain a boundary detection result, the method further comprises:
when the boundary detection result is a deviation, outputting position adjustment prompt information or sending a position adjustment instruction to the scanning device so that the scanning device can execute an adjustment action to adjust the position of the scanning area;
and the position adjustment prompt information is used for prompting a user to manually adjust the position of the scanning area.
4. The reading aid method of claim 1, wherein the scanning device comprises: the page number fixing mechanism and the mechanical arm are used for sending a scanning area switching instruction to the scanning device, and the page number fixing mechanism comprises:
under the condition that a medium to be scanned is a paper book, sending a force adjusting instruction to the page fixing mechanism so that the page fixing mechanism releases page turning limit of the paper book;
and sending a page turning control instruction to the mechanical arm so that the mechanical arm can pick up and turn pages of the paper book to switch the page number of the paper book.
5. The reading aid method of claim 1, wherein the scanning device comprises: the mechanical arm sends a scanning area switching instruction to the scanning device, and the scanning area switching instruction comprises the following steps:
and under the condition that the medium to be scanned is the electronic book, sending a sliding control instruction to the mechanical arm so that the mechanical arm can slide and turn pages of the electronic book to switch the page number of the electronic book.
6. The reading aid method according to claim 4 or 5, wherein the scanning device further comprises an identifier;
the sending of the scan area switching instruction to the scanning device previously further includes:
controlling the identifier to identify the type of the medium to be scanned, wherein the type of the medium to be scanned comprises: a paper book or an electronic book.
7. The reading aid method according to any one of claims 1 to 5, wherein the updating the text image to be read based on the scanning device includes:
acquiring a character image to be corrected in the scanning area based on the scanning device;
performing page number proofreading on the character image to be proofread to obtain a page number proofreading result;
and under the condition that the page number correction result is correct, updating the character image to be read based on the character image to be corrected.
8. An assistive reading device, comprising:
the acquisition module is used for acquiring character images to be read in the scanning area based on the scanning device;
the determining module is used for determining text information to be read based on the character image to be read;
the output module is used for outputting the voice data to be read based on the text information to be read;
and the sending module is used for sending a scanning area switching instruction to the scanning device so that the scanning device can switch the scanning area, and updating the character image to be read based on the scanning device.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the assistive reading method according to any of claims 1 to 7 are implemented when the processor executes the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the assistive reading method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210116492.5A CN114550174A (en) | 2022-02-07 | 2022-02-07 | Reading assisting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210116492.5A CN114550174A (en) | 2022-02-07 | 2022-02-07 | Reading assisting method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114550174A true CN114550174A (en) | 2022-05-27 |
Family
ID=81673441
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210116492.5A Pending CN114550174A (en) | 2022-02-07 | 2022-02-07 | Reading assisting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114550174A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115394282A (en) * | 2022-06-01 | 2022-11-25 | 北京网梯科技发展有限公司 | Information interaction method and device, teaching platform, electronic equipment and storage medium |
| CN116630982A (en) * | 2023-05-16 | 2023-08-22 | 读书郎教育科技有限公司 | Scanning area positioning method based on AI dictionary pen |
-
2022
- 2022-02-07 CN CN202210116492.5A patent/CN114550174A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115394282A (en) * | 2022-06-01 | 2022-11-25 | 北京网梯科技发展有限公司 | Information interaction method and device, teaching platform, electronic equipment and storage medium |
| CN116630982A (en) * | 2023-05-16 | 2023-08-22 | 读书郎教育科技有限公司 | Scanning area positioning method based on AI dictionary pen |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6377925B1 (en) | Electronic translator for assisting communications | |
| CN114550174A (en) | Reading assisting method and device | |
| CN108806719A (en) | Interacting language learning system and its method | |
| AU2017371714B2 (en) | Learning tool and method | |
| Quene et al. | Phonetic similarity of/s/in native and second language: Individual differences in learning curves | |
| Amengual | The acoustic realization of language-specific phonological categories despite dynamic cross-linguistic influence in bilingual and trilingual speech | |
| De Zoysa et al. | Project Bhashitha-Mobile based optical character recognition and text-to-speech system | |
| Yin | Training & evaluation system of intelligent oral phonics based on speech recognition technology | |
| JP3930402B2 (en) | ONLINE EDUCATION SYSTEM, INFORMATION PROCESSING DEVICE, INFORMATION PROVIDING METHOD, AND PROGRAM | |
| CN107203539B (en) | Speech evaluating device of complex word learning machine and evaluating and continuous speech imaging method thereof | |
| Scarborough et al. | Out of sight, out of mind: The influence of communicative load and phonological neighborhood density on phonetic variation in real listener-directed speech | |
| KR20140075994A (en) | Apparatus and method for language education by using native speaker's pronunciation data and thought unit | |
| CN204856534U (en) | System of looking that helps is read to low eyesight based on OCR and TTS | |
| Ravi et al. | Raspberry pi based smart reader for blind people | |
| KR20140079677A (en) | Apparatus and method for learning sound connection by using native speaker's pronunciation data and language data. | |
| CN114120769A (en) | Braille reading method, device, storage medium and electronic device | |
| WO2021231050A1 (en) | Automatic audio content generation | |
| Madhusha et al. | Mobile Base Sinhala Book Reader for Visually Impaired Students | |
| Rauf et al. | Urdu language learning aid based on lip syncing and sign language for hearing impaired children | |
| Bhatt et al. | Reading Assistant: a reciter in your pocket | |
| Gawande et al. | Novel Machine Learning based Text-To-Speech Device for Visually Impaired People | |
| Vonessen et al. | Comparing perception of L1 and L2 English by human listeners and machines: Effect of interlocutor adaptations | |
| Gopi et al. | Virtual Learning Environment for Visually Impaired People using OCR and TTS | |
| KR20240033676A (en) | Display-based communication system | |
| Kapgate et al. | Raspberry Pi Based Book Reader For Visual Impaired People |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |