TWM575595U - E-book apparatus with audible narration - Google Patents

Abstract

The invention relates to an automatic reading apparatus that receives a multi-media content including text content. The reading apparatus includes a display having a display region for the multi-media content; an input interface receiving an input signal associated with a location identification and/or a change of the multi-media content in the display region; and a reading and highlight unit generating audio content as well as one or more dynamic highlights associated with the text content, said dynamic highlight skip from a first portion of said text content to a second portion of said text content in response to the input signal.

Description

E-book voice reading device

The present invention relates to an electronic book reading device and a method thereof, and more particularly to an electronic book reading device capable of voice reading and dynamic marking of text content.

E-books have been in development for many years, and common e-book formats include PDF, EPUB, mobi, and AZW. According to the existing technology, the picture content and the text content contained in the e-book can be completely visually presented, but the reading function of the e-book is slow to develop, especially for the automatic reading function of machine learning. The reason is that the difficulty of reading aloud is quite high, and it is necessary to overcome the monotonous machine pronunciation and contextual analysis to be able to read smoothly. For example, the text content "3/4 Opening Ceremony" and "3/4 Impact Range", although both contain "3/4", the former is read aloud for March 4th and then read aloud for four. Three points. However, these problems have been gradually overcome with the development of AI technology, and the reading function of e-books will gradually become popular in the future.

The existing e-book reading device can open the reading function, and some of them are accompanied by the highlight of the text content to guide the reader to read, so that the reader can enter the reading state more easily through the cooperation of the mark of the text and the reading of the reading. However, the reading and marking functions of the existing e-books are only monotonously performed in accordance with the order of the text content, and the objects that are not allowed to be read and marked are arbitrarily selected.

Accordingly, it is necessary to develop a reading device or method that allows the target of reading to be selectively changed in accordance with the user's operation, and the marks of the characters are also synchronized.

The purpose of the present invention is to provide an automatic reading device configured to receive and display a multimedia content, the multimedia content comprising at least text content, the reading device comprising: a display having a display area for displaying a portion of the multimedia content; The interface receives an input signal associated with a location in the display area and/or a change in the portion of the multimedia content in the display area; and a reading and marking unit configured to generate Corresponding to the sound content of the text content and one or more dynamic tags, the dynamic tag jumping from a first portion of the text content to a second portion of the text content to respond to a input signal, wherein The first portion of the textual content is associated with a first sound content, and the second portion of the textual content is associated with the input signal and appears in the display area.

In a specific embodiment, the dynamic tag has a sentence tag. The dynamic tag has a single word tag. Alternatively, the dynamic tag has a sentence tag and a single word tag, and the sentence tag is visually separably overlapped with the single word tag. The range of the sentence tag is defined by two punctuation marks of the text content.

In a specific embodiment, the second portion of the textual content is associated with a second sound content.

The present invention also provides a non-transitory computer readable medium, comprising a plurality of instructions, executable by a processing unit to: analyze a text content contained in a multimedia content to identify a plurality of sentences and/or words; receive an input signal, Input signal and a location in a display area and/or Having a portion of the multimedia content related to a change in the display area; generating one or more dynamic indicia associated with the textual content to respond to the input signal, the dynamic indicia being viewable in the display area; The dynamic tag jumps from a first portion of the text content to a second portion of the text content, wherein the second portion of the text content is associated with the input signal and appears in the display area.

In a specific embodiment, the instructions are further executed: generating a corresponding sound content based on an identification of the sentence and/or the word of the text content, the output of the sound content being synchronized with the dynamic mark.

In a specific embodiment, the generating one or more dynamic tags associated with the text content comprises canceling an original dynamic tag.

In a specific embodiment, the change of a portion of the multimedia content in the display area includes a scrolling operation or a page turning operation with respect to the display area.

An automatic reading method is performed by a processing unit of an computing device, the method comprising: obtaining and displaying a portion of a multimedia content on a display area of a display, wherein the multimedia content has text content; starting a mechanical reading means based on The text content outputs sound content; generating one or more dynamic markers in the display area, the dynamic mark indicating a sentence and/or a word of the text content, the text content indicated by the dynamic mark and the Synchronizing the textual content associated with the sound content; and receiving an input signal associated with a location in the display area and/or a change in the portion of the multimedia content in the display area, the dynamic marker And outputting the sound content from a first portion of the text content to a second portion to return a signal, wherein the first portion of the text content is associated with a first sound content, the text content The two parts are related to the input signal and a second sound content and appear in the display area.

In one embodiment, the generating one or more dynamic markers in the display area includes simultaneously generating a first dynamic mark indicating a sentence and a second dynamic mark indicating a single word, the first dynamic mark and The second dynamic mark vision may overlap in a spaced apart manner.

In one embodiment, the input signal is associated with an operation of a touch interface, an image recognition result, or a speech recognition result.

In a specific embodiment, the display of the dynamic mark jumps from the first portion of the text content to the second portion to respond to the input signal, including the display of the dynamic mark from the first part of the text content A sentence jumps to the second sentence of the second part of the text content.

100‧‧‧ system

102‧‧‧Server

1020‧‧‧Central Processing Unit

1022‧‧‧Remember

1024‧‧‧Internet interface

1026‧‧‧Digital storage unit

104‧‧‧User terminal device, user device

106‧‧‧Network

200‧‧‧User device

202‧‧‧Processing unit

210‧‧‧Computer readable medium

220‧‧‧Internet interface

230‧‧‧ memory

231‧‧‧ operating system

232‧‧‧Content Playback Module

233‧‧‧Contents

240‧‧‧Output/Input Interface

250‧‧‧Reading and marking unit

260‧‧‧Output unit

270‧‧‧ input unit

300‧‧‧Reading and marking unit

301‧‧‧Text Generator

302‧‧‧Word Processing Engine

303‧‧‧Semantic Analysis Engine

304‧‧‧Optical Matching Engine

305‧‧‧Text Markup Engine

306‧‧‧Synchronous generation engine

400‧‧‧Display screen

401‧‧‧Display area

402‧‧‧Window

403‧‧‧ scrolling operation

404‧‧‧Page turning operation

405‧‧‧First choice operation

406‧‧‧Second selection operation

407‧‧‧ Third choice operation

501‧‧‧ dynamic mark

502‧‧‧ Dynamic Marking

503‧‧‧ paragraph mark

S600-S640‧‧‧Steps

S700-S730‧‧‧Steps

S800-S830‧‧‧Steps

The first image shows a system provided by this creation.

The second figure shows an embodiment of the first figure user device.

The third figure shows an embodiment of the present reading and marking unit.

The fourth figure illustrates a display screen of a display.

The fifth through fifth figures show schematic representations of various embodiments of the present creative dynamic indicia.

The sixth diagram shows the interaction process between the user and the automatic reading device.

The seventh diagram shows the flow of steps in the dynamic tagging of text content.

The eighth diagram shows the flow of steps in the creation of the reading method.

In the following detailed description of various exemplary embodiments, reference is made to the accompanying drawings, which form a part of the present invention. The embodiments are shown by way of example, and the specific embodiments described herein can be implemented by the example. Provide enough detail to enable technicians in the field to It is to be understood that the specific embodiments are described, and other embodiments may be utilized and other changes may be made without departing from the spirit and scope. In addition, although this may be the case, reference to "a particular embodiment" does not require a particular embodiment of the same or singular. Therefore, the following detailed description is not to be taken in a limiting

In the context of the overall application and the scope of the patent application, the following terms have the meanings explicitly associated with this unless the context clearly dictates otherwise. As used herein, the <RTI ID=0.0>"or" </ RTI> </ RTI> is used to include an "or" usage and is equivalent to the term "and/or" unless specifically stated otherwise. The term "subject" is not exclusive and expressly relies on the majority of other factors not recited, unless the context clearly dictates otherwise. In addition, in the overall application, the meaning of "one", "one" and "the" includes plural references. The meaning of "in" includes "in" and "in".

As used herein, the term "network connection" means a collection of links and/or software components that enable a computing device to communicate with another computing device over a network. One such network connection can be a Transmission Control Protocol (TCP) connection. A Transmission Control Protocol connection is a virtual connection between two network nodes and is generally established through a Transmission Control Protocol handshake protocol.

A brief summary of these innovative topics is provided below to provide a basic understanding of certain aspects. This short narrative is not expected as a complete overview. This short description is not intended to identify primary or critical elements, or to depict or limit the range. Its purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

The first figure shows a system (100) provided by the present invention, which comprises one or more servers (102) and a plurality of user terminal devices (104) connected thereto, also called user devices or uses. The device, in particular the terminal device (104), is suitable as an e-book reader. For the user device (104), the server (102) is a remote server. The server (102) can be programmed to create a website or other form accessible by the browsing device of the user device, so that the user can download the data, such as the application, that the server is allowed to access via the network (106). , multimedia content materials, software updates, and so on. The server (102) can be further configured to perform a particular operation and provide the results of the operation to the user device (104) via a network connection. In some embodiments, the server (102) can provide a download link for an e-book website and an e-book reading software. The user can access the e-book via a given server, and the server's access can be restricted, such as limiting the number of people accessing or data traffic.

In general, the server (102) includes one or more central processing units (1020) and memory (1022) for storing a plurality of operational instructions executable by the processor. A network interface (1024) network is coupled to one or more networks (106) and user terminal devices (104) for receiving data, requests, and commands from the network and transmitting various forms of data to the user device, such as The multimedia content data stored in the digital storage unit (1026) includes image data, text data and sound data. The processor (1020) can receive information and instructions from other computer systems or services via the network (106). For example, the processor (1020) can utilize the web interface (1024) to receive or provide various content items for e-book presentation. The processor (1020) can further receive or transmit synchronization information about the content item using the web interface (1024). The processor (1020) can communicate with the memory (1022) to access various operational instructions therein. In some embodiments, the role played by the central processor (1020) and the operations performed thereby may be shared or replaced by processors inherent to the user device (104).

The memory (1024) contains a plurality of computer executable instructions that are executable by the central processing unit (1020) to carry out the various operations disclosed herein. The memory (1024) may comprise any combination of transient or non-transitory memory, including RAM, ROM, hard disk, solid state hard disk, flash memory, and the like. The memory (1024) can store an operating system that provides a plurality of computer program instructions for use by the processor (1020) for general content management and operation. In other embodiments, the digital data storage unit (1026) can be included in the configuration of the memory (1024) and store various content items for presentation at the user device, such as picture content, text content, and sound content. The digital data storage unit (1026) may contain other information about the content item, such as synchronization mapping information about the content item, metadata of the content item, and the like. In other embodiments, the server (102) can be networked to another external digital data storage unit (not shown) to retrieve content items for presentation at the user device or stored in memory (1024).

The user terminal device or user device (104) can be a personal computer, a tablet computer, a number of person assistants, a mobile device, or any suitable form. In this embodiment, the user device (104) includes a display, an input unit (such as a physical keyboard, a touch screen, a mouse, a radio, or an imaging unit), a processing unit, a memory, and the like to perform the creation. Configuration of all embodiments.

The second figure shows an embodiment (200) of the first user device (104), which includes a processing unit (202), a computer readable medium (210), a network interface (220), and a memory. (230), an output/input interface (240) and a reading and marking unit (250). Similarly, the processing unit (202) can receive various information, instructions, and multimedia content from the network. The processing unit (202) can utilize the web interface (220) to receive various content items for presenting the e-book. The processing unit (202) further utilizes the web interface (220) to transmit or receive other information or instructions for performing various embodiments of the present authoring, such as synchronization mapping information for a plurality of content items. The processing unit (202) can access a computer executable program included in the memory (230), such as an operating system (231) of the client device, a content playing module (232), and content data (233), and input/ The output interface (240) is output to an output unit (260), which can include one or more output devices, such as a display and a speaker, for presenting various content items to a user. The input/output interface (240) can receive input from an input unit (270) The input unit (270) may include one or more input devices such as a touch screen (a combination of an output device and an input device), a mouse, a microphone, and an imaging device. In the case of a touch screen, a touch event generates a corresponding input signal, and the processing unit (202) can determine one or more coordinates about the touch event according to the input signal, and further according to the coordinates The analysis processing unit (202) can recognize one or more pixels of the display and corresponding touch behaviors. Accordingly, the processing unit (202) can determine the associated pixel output based on the coordinates.

The memory (230) can include any combination of transient and non-transitory, such as RAM, ROM, hard disk, solid state hard disk, and flash memory. The operating system (231) provides computer programming instructions for the general management and operation of the user device, which vary from one type of client device and are well known in the art and will not be described herein. The content play module (232) can be configured to perform presentations on various content items, as well as provide a user interaction interface for controlling content playback. The content material (233) contains one or more content items, such as text content, picture content, and sound content, which can be played via the content playback module (232). The content material (233) may further include other information related to each content item, such as synchronization mapping information between two different content items, and metadata of each content item. The content material (233) can be updated based on other content material received from the web interface (220) or the output/input interface (240). The user device (200) can acquire other external content items and store them in the content material (233) for real-time streaming playback or playback at any time. In the case of an e-book, the content playing module (232) can process text content, picture content, and sound content, and the content playing module (232) can also provide user operating elements such as a page turning button or a scrolling component.

The reading and marking unit (250) is configured to perform an audible reading and marking action of the corresponding textual content. In other embodiments, the reading and marking unit (250) can be split into a reading unit and a marking unit that are independent of each other. In some embodiments, the reading and marking unit (250) may be internal A portion of the playback module (232) or associated extension. Alternatively, part of the work of the reading and marking unit (250) can be performed at the server (100) side. The third figure shows an embodiment (300) of the creation reading and marking unit, comprising a text generation engine (301), a word processing engine (302), a semantic analysis engine (303), an audio matching engine (304), and a text marking engine. (305) and a synchronization generation engine (306).

The text generation engine (301) is configured to recognize textual content from one or more content items stored in the server (100) or content material (233) and to generate visual text and punctuation that can be displayed, and based on the content item Other information in the game determines visual effects such as text layout, fonts, and fonts. The layout may include a visual representation of the picture and the text. The content items may be defined by various e-book-specific formats, such as PDF, EPUB, and AZW. The visual text may be included in a single page of the e-book or in a space of multiple pages. In other possible embodiments of the present creation, such as in an audiobook application, the text generation engine (301) can be configured to generate corresponding textual content from the received sound content using known speech recognition means.

The word processing engine (302) is configured to recognize one or more sentences in the textual content. For example, according to known rules, a sentence can be a text between two adjacent periods, or a text between any two adjacent punctuation marks (comma and period). One or more of the text components covered by the brackets can also be considered as the identification of the sentence. In other embodiments, the identification of sentences may be further optimized in accordance with techniques based on mechanical learning, which may resolve unrecognizable errors that may result from punctuation errors. In one embodiment, the text content recognized as a sentence may be given an identifier or label and stored in the memory along with the corresponding text content, ie each sentence has a respective identifier or label. For example, specific identifiers for these sentences can be given so that sentences can be recognized and the relationship between sentences and sentences can be clearly defined, such as the order relationship of sentences and sentences, the number of paragraphs or lines in which sentences appear.

The semantic analysis engine (303) is configured to determine semantic features associated with the one or more sentences based on the identified one or more sentences, which may be stored in the memory along with the textual content and the identifier or tag. The semantic feature described here refers to the statistics or measurement related to the grammar, meaning and/or composition of the sentence. In an embodiment, the semantic analysis engine (303) may divide the text of each sentence into segments and determine corresponding semantic features for each segment. The semantic analysis engine (303) can be implemented by known machine learning means, and the establishment of the semantic analysis engine (303) can be done at the remote server and downloaded and installed to the user terminal device. Alternatively, the semantic analysis engine (303) may not execute in the user device, but instead execute at the remote server and store the analysis results at the remote server. The Semantic Analysis Engine (303) continuously optimizes the accuracy of semantic analysis and even detects errors in sentences through continuous training feedback.

The audio matching engine (304) is configured to recognize one or more sound content corresponding to the sentence based on the semantic features associated with the sentence, thereby completing the matching of the text content and the sound content. The sound content may be composed of one or more files and may be converted to an audio signal for output via a speaker. In one embodiment, the audio matching engine (304) can access an audio sample database (not shown) that can store candidate sound content corresponding to various words. In an embodiment, in the audio sample database, each sound content item corresponding to a word or a word may be associated with one or more semantic features, and the matching is based at least on words, words, and/or sentences. The semantic features and the identification of the semantic features associated with the sound content. The matching causes the textual content (words, words, sentences) to be associated with one or more sound content. The audio matching engine (304) can be implemented by known means, such as automatically reading an application. In other embodiments, for the application of the audio book, the pre-recorded human voice can read the sound content synthesized by the audio matching engine, that is, the voice content read by the human voice can be processed to be associated with the corresponding text content for playing.

The text tag engine (305) is configured to generate one or more sentences and/or words that are dynamically tagged with the start signal or the input signal based on a start signal or an input signal. The dynamic indicia can be visually presented to the user via the display. Dynamic markups have any possible form, such as fluorescent marks on text, bottom lines below text, color/font/fonts of text, and so on. The dynamic here refers to the action that the mark appears and jumps between sentences and/or words as the reading target progresses during automatic reading. When the automatic reading stops, the mark will be stationary or disappear. The start signal indicates the beginning of an automatic reading operation. The text tag engine (305) marks the first sentence or first word identified in the text content to echo the start signal. Alternatively, the start signal may further indicate the beginning of the automatic reading operation after the pause. The frequency of the mark jump is related to the length of the sentence or word and the speed of automatic reading. The input signal is generated via an input unit (270), where the input signal indicates location information on a display area of the display. In one embodiment, the input signal is generated based on identification of a target or a set of labels in a display area of the display (eg, the user clicks on the touch screen), wherein the coordinates are associated with one or more Pixel position. In another embodiment, the input signal is generated based on a selection of a portion of the multimedia content (eg, the user clicks on a portion of the display content in the display area). Alternatively, the input signal indicates location information of the multimedia content that is not displayed (eg, the user clicks on the third chapter in the displayed directory). The text tag engine (305) can associate a tag to the undisplayed text content to return the input signal. It is worth noting that although the multimedia content is not displayed, a location information of each content item in the multimedia content may be determined according to the typesetting rules that have been applied (eg, the third sentence of the third paragraph of the text content is on the ninth page first. To the fifth line). The content playing module (232) can provide a navigation field for inputting an e-book, such as a chapter, a page number, and a line number.

Taking an electronic book as an example, the fourth figure illustrates a display screen (400) of a display in which a display area (401) displays a portion of the text content while other portions are not displayed or are covered by the window (402). Alternatively, the undisplayed content may appear via a scrolling operation or a page turning operation. For example, the input signal corresponding to the scrolling operation (403) indicates that the set of labels on the display area (401) changes along a longitudinal direction, whereby the undisplayed text content can be above the display area (401). Or load the screen below. The input signal corresponding to the page turning operation (404) indicates that the label set on the display area (401) changes along a lateral direction, whereby the undisplayed content can be loaded into the screen by the left and right sides of the display area (401). . The dynamic mark may not respond to the loading of the undisplayed content, or the dynamic mark may be maintained in the display area (401) in response to the loading of the undisplayed content. A selection operation can be associated with the text content in the display area (401). As shown, the input signal corresponding to a first selection operation (405) indicates a label or a set of labels corresponding to the text content "I know", and the text markup engine can mark the sentence or word corresponding to the text content accordingly. . The input signal corresponding to a second selection operation (406) indicates a label or a set of labels corresponding to a blank margin of the page, and the text markup engine can mark the sentence of the text content closest to the margin of the margin or Words. The input signal corresponding to a third selection operation (407) indicates a label or a set of labels corresponding to the two sentences, and the text markup engine can select one of the two sentences accordingly.

Various forms of input units can perform the aforementioned operations and generate corresponding input signals. The touch screen is a universal input unit that also provides intuitive operation. Alternatively, the input unit may be used by the imaging device to capture the eyeball image or the gesture image of the reader, and cooperate with the image recognition to generate an input signal corresponding to the operation. Known image recognition techniques can determine the position or panning of the reader's eye nitrile in the display area (401) to identify the aforementioned operations. For example, the selection operation can be recognized when the eyeball is staring at a position in the display area (401) for a time or in conjunction with a lash action. When the reader is away from the display and When a waving gesture or a pointing gesture is given, a page turning operation or a selection operation can be recognized. Alternatively, the input unit may be a microphone for capturing human voice. Known speech recognition techniques can determine the keywords given by the reader and correspondingly generate input signals for associated selection operations. Further, in conjunction with the search technology of known search techniques, the text tagging engine can tag all selected keywords in the text content. The selectivity of these input units is friendly to the disabled and on the other hand to the teaching field, not just to the use of known e-books.

Alternatively, one or more dynamic markers can be displayed in the display area. Figures 5 through 5B show various embodiments of the present dynamic tag. Figure 5A shows a dynamic marker (501) for a single sentence. In the embodiment of the volume animation surface, the dynamic mark (501) can be maintained at a level or a range of the display area during automatic reading, so that the picture is dynamically scrolled as the automatic reading is performed. In an embodiment of the page flipping screen, when the dynamic tag appears at the bottom of the current page, then the loading of the next frame will be dynamically tagged to the top content of the screen. Figure 5B shows a dynamic markup (502) for a single text. However, when a lengthy sentence is encountered or the reading speed is too fast, simply using the sentence mark (501) or the single word mark (502) has its disadvantages. Therefore, combining the two can make up for each other's shortcomings. As shown in the fifth C-picture, two types of marks are simultaneously displayed, wherein the single-word mark (502) is included in the single-sent mark (501), and both can give appropriate visual distinction, such as color or transparency processing. The fifth D diagram further shows a paragraph mark (503) that is suitable for narrowing down the content based on the paragraph.

Referring back to the third diagram, the synchronization generation engine (306) is configured to synchronize the identified sound content with the corresponding one or more dynamic markers. The synchronization generation engine (306) can output the sound content of a sentence or a word and the dynamic mark to the output unit (270), such as a speaker and a display, according to the stored synchronization information or identification information (ie, an identifier or a tag). In an embodiment, the synchronization generation engine (306) may utilize a known identifier or tag to place a portion of the textual content and its corresponding dynamic The association is associated with a portion of the sound content, such as via a known linking means, wherein the identifier or label is used to identify a sentence or word in the textual content. In some embodiments, the synchronization generation engine (306) can continue to perform the linking means until all textual content and sound content are completed in synchronization. Execution of the synchronization generation engine (306) can be done at the remote server and the synchronization results stored in the cloud, which can be downloaded to the user device along with the multimedia content. Further, the synchronization generation engine (306) may determine the display time of the synchronized dynamic mark based on the play time of the sound content item and record it in the synchronization result.

The sixth diagram shows a flow diagram of the interaction of the user with the automatic reading device (e.g., user device of the first figure, 104), including steps S600 through S640. In step S600, the user turns on the electronic book reader installed by the user device and opens the electronic book file via the reader. The e-book reader can be downloaded from a website or link provided by the remote server, or can be built into the user device. The reader can include a control interface to allow the user to selectively navigate the content of the article. The reader can also include other additional features, such as automatic reading and text markup. The reader can provide a reading window displayed on a display area of the display, which displays a portion of the e-book content, including text content and image content, and even clickable links. The navigation mode of the reader can be scrolling navigation mode or page turning navigation mode according to the type of user setting or loading the e-book file. After the reader is opened, part of the content of the article appears in the display area, and step S600 ends.

In step S610, the user initiates an automatic reading and marking function via a control interface (such as a virtual or physical button) of the reader. In one embodiment, the user can click on the virtual button in the fourth view window (402) to call the display of a selection interface. Alternatively, the user can pull out a selection interface listing a plurality of selectable functions in a sliding gesture from an edge of the display area. The reading and marking functions can be individually independent. Preferably, when both are determined to be active, the result of reading and marking should be Synchronize. When the reading and marking function is active, when the user presses the play button, the sound signal and the dynamic mark read aloud are substantially received by the user at the same time. The dynamic mark appears in the display area in units of words, words, or single sentences, and the text associated with the dynamic mark matches the text associated with the sound signal completely or partially. Dynamic markers and audio signals are generated at an appropriate speed and in the order in which the text content is predetermined. As shown in the fourth figure, the dynamic mark and the spoken sound signal are automatically and synchronously associated with the last sentence or the last word in the display area by the first sentence or the first word in the display area. In the scrolling navigation mode, when reading the last sentence or word in the display area, the reader can automatically scroll the animation surface, replacing the undisplayed text content with the original part, and reading and marking from the updated part. In the page navigation mode, when reading the last sentence or word in the display area, the reader can automatically load the undisplayed text content to replace the original text, and then read and mark from the updated part. Unless the user instructs the reader to stop the action, the sound signal and the dynamic mark read aloud will continue to play in the order of the article until the end of the article, and the process ends in step S610.

At step S620, the reader determines whether the user has indicated that the jump or skips part of the article content. The jump or skip refers to the user specifying a new mark in the article and reading the target content, which is not included in the target content currently marked and/or read. The skipping or skipping action may represent that the user wishes to switch from the current reading, marking and/or display of the target content to another unread, unmarked and/or undisplayed target content, ie the user wishes to change the current reading and / or mark the text content. For example, a user can determine and change spoken and dynamically tagged sentences via a reader. The reader continuously detects whether there is any input signal indicating that the user has jumped the article, and ends step S620.

If the reader does not receive the relevant instructions, the reader will continue to read and mark the next sentence or word in the order of the predetermined article until the end of the article (step S630).

At step S640, the user instructs the reader to switch the target content of the automatic reading and dynamic marking. For example, the user can navigate through the target content and select a starting location for automatic reading and dynamic tagging therein. In a possible embodiment, during the navigation process (whether scrolling or page turning), the automatic reading and dynamic marking actions automatically identify a starting position as the text content in the display area changes. For example, when the process of scrolling the animation surface causes the current reading and the text in the mark to disappear in the display area, the reader can be configured to automatically identify a position in the current display area that allows new readings and indications to continue from there. In other embodiments, the reading and marking actions do not change with the change of the display area, that is, even if the process of scrolling the animation surface causes the current reading and the characters in the mark to disappear in the display area, the position and order of the reading are still Not changed. This applies to users who only navigate, but are not intended to change the current reading and marking goals.

The seventh figure shows the flow of steps for dynamically creating the text content of the present creation, including steps S700 to S730. These steps can be implemented by a plurality of executable instructions stored in one or more memories, such as the presence of a first map server or user device. The execution of these steps can be done at the user terminal device, or a portion of these steps can be performed at the remote server, or portions of these steps can be performed jointly by the terminal device and the remote server.

At step S700, an instruction can be configured to perform analysis of a multimedia content (such as an e-book archive) to identify a plurality of sentences and/or words of the textual content of the multimedia content. The identification can be implemented based on known semantic recognition and mechanical learning. In an embodiment, the recognition of the sentence is based on an association between punctuation marks. In other embodiments, the identifying may include attempting to cut a different segment of text content for analysis. In some embodiments, the identifying may include analyzing the semantics of a series of textual content to determine the extent of a sentence. The recognized sentence, word or paragraph can be given corresponding identification information, such as an identifier or a label, which can specifically indicate that the sentence is in an article. Context or location. The identification information can be stored and transmitted along with the multimedia content, and step S700 is ended.

In step S705, an instruction is configured to perform recognition of the sentence and/or the word based on the aforementioned text content to generate corresponding sound content, which may be implemented by known mechanical reading means. Known means can output or synthesize the corresponding sound content for each word, word and single sentence of the text content. These sound content can be generated and stored in appropriate memory before reading aloud, or the sound content can be generated during the execution of the reading and output in time.

In step S710, an instruction is configured to cause a processing unit to process an input signal, in particular, the input signal and a location in a display area of a display and/or a change in a portion of the multimedia content in the display area. related. The input signal is received by the output/input interface (240) of the client device as shown in the second figure. Different input units (260) can be in communication with the output/input interface (240), such as a touch panel, an optical lens or a microphone. The input signals generated by these input units are as described above. As shown in the fourth figure, the user can generate a navigation operation and a selection operation with respect to the display area via the input unit, wherein the navigation operation (403, 404) will cause a change in the portion of the multimedia content in the display area ( The selection operation (405, 406, 407) is to cause a position selection in the display area. In one operation, the user first performs a navigation operation to find the content that he or she wishes to read, and then performs a selection operation to determine a reading item, such as a sentence or a word. Accordingly, the processing unit obtains at least one or more location information in the display area, and step S710 ends.

At step S720, an instruction is configured to cause the processing unit (or the tag unit) to generate one or more dynamic tags associated with the textual content, the dynamic tags being viewable in the display area. In an embodiment, the processing unit automatically generates the text dynamically tagged in the display area. In another embodiment, the processing unit finds the location identification information in the display area obtained as described above. Corresponding to the position-recognized sentence or word and generating the dynamic mark in the corresponding range in the display area. As shown in the fourth figure, regardless of whether the position associated with the user's selection operation (405, 406, 407) directly indicates a sentence or a word of the article, the sentence most relevant to the position should be preferentially identified and given a mark. In an embodiment, the processing unit may further generate, according to the navigation operation, the text content that the dynamic mark ends in the display area. For example, according to a page turning operation, a new dynamic tag can be generated from the first sentence of the new page. One or more visible dynamic markers can appear in the display area. Figures 5A through VD show schematic representations of single sentence marks, single word marks, and combinations thereof.

In step S730, the processing unit causes the dynamic mark to jump from a first portion of the text content to a second portion of the text content to return a signal, wherein the second portion of the text content appears on the display In the area. In a possible case, a first sentence in the current display area has seen a dynamic mark, and after the user selects a second sentence in the current display area (ie, generates an input signal), the original first sentence The dynamic marker jumps to the second sentence selected by the user. The jump described herein does not refer to a specific jump action, but rather to a visual effect that is visually similar to jumping or switching. The visual effect of the jump can be seen that the dynamic marker ignores the stay of other sentences between the first sentence and the second sentence. In another possible case, a first sentence with a dynamic mark disappears from the current display area due to the navigation operation, and the user selects a second sentence in the current display area (ie, generates an input signal). The dynamic marker returns to the display area and marks the second sentence. Although the dynamic mark may disappear from the display area due to the operation, it is assumed that the dynamic mark is a content that jumps from the undisplayed portion to the display. Step S730 may be performed simultaneously with step S720 or as part of step S720.

The eighth figure shows the flow of steps of the present reading method, including steps S800 to S830. These steps may be performed by one or more computing devices (such as the first image server 102 and the user device 104). Or jointly. For example, when the user device is in communication with the server, portions of these steps can be performed jointly by both. Alternatively, in the detached state, the user device can perform these steps independently.

In step S800, a multimedia content is obtained via a remote server or a user device, and a portion of the multimedia content is displayed on a display area of a display. The multimedia content may be a combination of various content items, such as text content, picture content, sound content, and video content, which may also be integrated in the streaming content from the remote server to the user device. The display can be included in the user device or an external device that is independent of and in communication with the user device. The display area (such as the fourth figure, 401) displays a portion of the multimedia content. In the case of an e-book, the display area is mainly text content and can be interspersed with pictures or billboards between the texts. Step S800 ends.

In step S810, a mechanical reading means is initiated via the user device to output an audio signal based on the text content. The user device can be configured to have the ability to read text. For example, the user device can include an e-book reader, a sound database, a semantic recognition engine or module, and a speaker. The sound data stock is provided with sound data corresponding to each word, word or sentence, and the data can be matched with the result of the semantic recognition to output a corresponding sound signal. The audio signals described herein may be in the form of digits or analogs, and are not limited to audible signals in the circuit transmission phase or the final output. In the case where the user does not specify, mechanical reading can start from any part of the text content, such as the first word of the text content, or the first word of the current text content in the display area, or the position where the previous reading ends. The reading continues until the end of the article or the user actively stops, ending step S810.

At step S820, one or more dynamic markers are generated in the display area via the user device, the dynamic markers indicating a sentence and/or a word that is desired to be viewed by the reader. The text content indicated by the dynamic mark is synchronized with the text content associated with the sound signal. Synchronization as described herein means that the dynamic marker produces the same range as the currently spoken word or the same sentence as the currently spoken word. It is not limited to the same time when the relevant signals occur. In one embodiment, the dynamic tag is a sentence tag that appears in a position where the sentence is displayed such that the sentence is visually distinguishable from other words (as in Figure 5A). In another embodiment, the dynamic indicia is a single-word indicia that appears in a single-word display such that the word can be visually distinguished from other text (as in Figure 5B). The single word mark can essentially jump to the next word at the speed of reading. In other embodiments, the dynamic mark is a combination of a sentence mark and a single word mark, which simultaneously appears at a position of a sentence and a single word position in the sentence, and the two are visually distinguishable (eg, the fifth C picture) ). For example, the sentence mark and the single word mark may each have a different color, or one of them may be a text bottom line. The dynamic mark will continue to jump to the end of the article until the reading stops, ending step S830. Preferably, step S810 is performed together with step S820.

In step S830, before the reading and the dynamic mark have not stopped or after the stop, an input signal is received via the user device, the input signal is recognized with a position in the display area and/or the part of the multimedia content is displayed on the display. Related to changes in the area. The user device can include or be communicatively coupled to an input unit, such as a touch panel, optical lens or microphone, that allows the user to operate to indicate a location information on the display area and to allow the user to navigate through all of the multimedia content in the display area. The location information includes a set of landmarks or a set of labels on the display area that may indicate one or more user operations, such as the aforementioned selection operations or navigation operations. The display of the dynamic mark and the output of the audio signal jump from a first portion of the text content to a second portion to respond to the input signal, wherein the second portion of the text content appears in the display area. When a selection operation is recognized by the user device, a sentence or a word (most associated with the location information) is then identified in response to the selection operation. A sentence or list that is recognized based on the input signal becomes a new target for reading and marking, and is immediately read and marked. The target of the dynamic mark jumps from the original sentence (the first sentence) to the recognized sentence (the second sentence), the first sentence and the second sentence are different sentences, and Limited to the first sentence before the second sentence. If the input signal indicating a navigation operation causes the content of the display screen to change, the dynamic mark changes the position appearing on the display screen or disappears with the change. In an embodiment, when the dynamic tag thus disappears, the user device can generate a new dynamic tag in the changed display area to mark the current content while reading the target and synchronizing to the new target content. In some embodiments, such as an e-book directory containing chapter links or a shortcut key returning to the home page, an input signal indicating a selection operation or a navigation operation may cause the user device to direct the reader to the page associated with the selected chapter link, while Reading and marking the target are also synced to the page. Based on the input signal, the target of the dynamic mark and/or the read target jumps to the new target content, and step S830 ends.

To understand the steps and combinations of steps in the description of this type of flowchart, it can be used as a computer program instruction. These program instructions can be provided to a processor to make a machine, such that when the instructions are executed on the processor, a method for implementing the actions specified in the flowchart block or majority of the blocks is generated. The computer program instructions are executable by a processor to perform a series of operational steps by the processor to form a computer implemented program, such instructions being executed on the processor to provide for implementation in the flowchart The steps of the action specified in the block or most blocks. The program instructions can be stored on a computer readable medium or machine readable medium, such as on a computer readable storage medium.

Accordingly, these descriptions support a combination of means for performing such specific actions, a combination of a plurality of specific operations that support execution, and a program command mode that supports the execution of such specific actions. It will also be appreciated that each block in the flowchart depiction and the combination of blocks in the flowchart depiction can be implemented by a module, such as a system based on a special purpose hardware, which performs the specific actions. Steps, or a combination of special purpose hardware and computer instructions.

The above description provides a complete description of the manufacture and use of the combinations of the specific embodiments. Since many specific embodiments can be made without departing from the spirit and scope of the invention, these specific embodiments are intended to be included in the scope of the appended claims.

Claims (6)

An automatic reading device configured to receive and display a multimedia content, the multimedia content comprising at least text content, the reading device comprising: a display having a display area for displaying a portion of the multimedia content; an input interface receiving an input a signal, the input signal being associated with a location in the display area and/or a change in the portion of the multimedia content in the display area; and a reading and marking unit configured to generate an association with the text a sound content of the content and one or more dynamic tags, the dynamic tag jumping from a first portion of the text content to a second portion of the text content to return a signal, wherein the first portion of the text content In association with a first sound content, a second portion of the textual content is associated with the input signal and appears in the display area. The automatic reading device of claim 1, wherein the dynamic mark has a sentence mark. The automatic reading device of claim 1, wherein the dynamic mark has a single word mark. The automatic reading device of claim 1, wherein the dynamic mark has a sentence mark and a single word mark, and the sentence mark is visually separably overlapped with the single mark. The automatic reading device of claim 2, wherein the range of the sentence mark is defined by two punctuation marks of the text content. The automatic reading device of claim 1, wherein the second portion of the text content is related to a second sound content.