DE102012219852A1 - Method for manipulating text-to-speech output to operator, involves detecting gesture of operator in gesture information and evaluating gesture information to detect operator command, where parameter of text-to-speech output is adjusted - Google Patents

Abstract

The method involves detecting a gesture of the operator in the gesture information and evaluating the gesture information to detect a command of the operator. A parameter of text-to-speech output is adjusted using the operator command in order to manipulate the text-to-speech output. The gesture is detected as a spatial sequence of movements of a body part (118) of the operator within a detection area (116) of a detection device (102). Independent claims are included for the following: (1) a device for executing the method; and (2) a computer program product for executing the method.

Description

State of the art

The present invention relates to a method for influencing a text-to-speech output, to a method for outputting text as speech, to a corresponding device and to a corresponding computer program product.

The DE 10 2008 051 757 A1 describes a multimodal user interface of a driver assistance system for entering and presenting information.

Disclosure of the invention

Against this background, with the approach presented here, a method for influencing a text-to-speech output, a method for outputting text as a language, furthermore a device which uses one of these methods and finally a corresponding computer program product according to the main claims are presented. Advantageous embodiments emerge from the respective subclaims and the following description.

Information can reach an information recipient on different transmission paths. In particular, the information recipient can receive the information about his sense organs. In order to relieve the eyes, which are generally referred to as the most important sensory organs, it is possible to read out textual information. This allows the eyes to record other information.

The invention is particularly suitable for a so-called driver information system in a vehicle that outputs information to the driver.

If the reading aloud, that is to say an output of text as a language, is to be controlled by the information receiver, it is necessary to be able to transmit an instruction of the information receiver to a device for the output.

The approach presented here is based on the recognition that reading aloud text can be controlled by gestures or gestures of the information recipient without, for example, serving real or virtual buttons. The gestures can be executed by the information receiver freely in the room. A movement sequence of the gesture can represent a control command and / or a sequence of control commands. The gestures can be self-explanatory. The gestures can also be simple movements that are logically linked to an abstract control command.

One method of affecting a text-to-speech output to an operator comprises the following steps:
Detecting a gesture of the operator in a gesture information;
Evaluating the gesture information to recognize an instruction of the operator;
Adjusting at least one parameter of the text-to-speech output using the instruction to affect the text-to-speech output.

A method of outputting text as a language includes the following step:
Providing a speech signal using at least one parameter that is influenced by a method according to the approach presented here.

A text-to-speech output may be understood to mean a method of outputting text as a language. In the text-to-speech output, a continuous text can be converted into an acoustic speech output. For example, the text-to-speech output may read an electronic text. A gesture can be a distinguishable character with an associated meaning. The gesture can be a gesture. The gesture may in particular be a movement of the operator. The meaning can be derived, for example, from the movement. Likewise, the meaning can be assigned arbitrarily to a movement. For example, the gesture may be based on a sign language. Sign information may be, for example, an electronic signal. The sign information can be an image of the gesture. For example, the gesture information may represent a trajectory of movement of a body part of the person performing the gesture. An instruction may be a command or command to control an output of speech from a text segment. The instruction may be related to a content of the text. Likewise, the instruction may be related to the output of the text. A parameter may be, for example, a voice speed, a volume, a reference to a text position or a hierarchical level of the text. A speech signal may be an electrical signal for a loudspeaker or an acoustic signal from a loudspeaker. The speech signal can be created, for example, by synthesizing letters and / or syllables of words of the text into a signal representing a speech. The parameter may influence the synthesis and / or content of the synthesized text itself.

The gesture can be detected as a spatial movement sequence of at least one body part of the operator within a detection space of a detection device. For example, a head movement and / or a hand movement and / or a trunk movement can be detected. A detection room can be arranged freely in the room. The detection space can have a spatial extent in height, width and depth. A detection device can detect the gesture, for example, via ultrasound, via a video or stereo video recording, via electromagnetic waves and / or fields and / or via lidar.

The gesture information and a voice command of the operator may be evaluated in response to the voice command to recognize the instruction of the operator. A voice command can be an acoustic command from the operator. The output of the language can be controlled via both input options. Control over gestures and speech can provide fast and / or precise control.

The method may include a step of comparing the instruction with a previously recognized instruction of the operator to identify a sequence of instructions, wherein in the adjusting step, the parameter is further adjusted using the instruction sequence. For example, a sequence of instructions may be a combination of instructions. The instructions can be cumulated. For example, the instruction can concretize and / or supplement the previously recognized instruction.

The instruction may be recognized as an acceleration command, a deceleration command, a start command, a pause command, a forward command, a backward command, an up command, and / or a down command. Through a discrete number of commands each gesture can be assigned a unique command. The different gestures may be, for example, linear and / or circular movements of a body part, in particular a hand of the operator.

In response to the acceleration command, a text may be output faster than before the acceleration command. In response to the deceleration command, the text may be output more slowly than before the deceleration command. In response to the start command, the text-to-speech output can be started. In response to the pause command, the text-to-speech output may be interrupted. In response to the forward command, a text element following in chronological order and / or in sequence may be output. In response to the backward command, a temporal and / or sequential text element may be output. In response to the up command, a text element of a higher hierarchical level may be output than before the up command. In response to the down command, a text element of a lower hierarchical level may be output than before the down command.

The method may include a step of issuing an acknowledgment of the acknowledgment of the instruction. The confirmation can be made optically and / or acoustically. For example, the confirmation may be via a voice output or a beep. For example, one or more words may be output that reflect a context of the recognized statement. Likewise, the confirmation can be done via a signal light or a visual display.

The method may include a step of providing a name of a hierarchical level of a currently selected text segment, the name being provided in response to a beginning of the current text segment. A hierarchical level can be understood as a structural element of structuring the text. For example, a text may include a title, a subject, an author reference, a summary, and / or a plurality of paragraphs.

The present invention further provides an apparatus configured to perform the steps of one of the methods of the invention in corresponding devices. Also by this embodiment of the invention in the form of a device, the object underlying the invention can be solved quickly and efficiently.

In the present case, a device can be understood as meaning an electrical device which processes sensor signals and outputs control and / or data signals in dependence thereon. The device may have an interface, which may be formed in hardware and / or software. In the case of a hardware-based embodiment, the interfaces can be part of a so-called system ASIC, for example, which contains a wide variety of functions of the device. However, it is also possible that the interfaces are their own integrated circuits or at least partially consist of discrete components. In a software training, the interfaces may be software modules that are present, for example, on a microcontroller in addition to other software modules.

Also of advantage is a computer program product with program code which is stored on a machine-readable carrier such as a semiconductor memory, may be stored in a hard disk memory or an optical memory and used to perform any of the methods of any of the embodiments described above when the program product is executed on a computer or device.

The invention will now be described by way of example with reference to the accompanying drawings. Show it:

1 a block diagram of an apparatus for influencing a text-to-speech output according to an embodiment of the present invention;

2 a flowchart of a method for influencing a text-to-speech output according to an embodiment of the present invention;

3 a representation of a first gesture of an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention;

4 a representation of a second gesture of an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention;

5 an illustration of a third gesture of an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention;

6 a representation of a fourth gesture of an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention;

7 a representation of various hierarchical levels of a text for speech output according to an embodiment of the present invention; and

8th an illustration of various gestures of an operator for influencing a text-to-speech output according to an embodiment of the present invention.

In the following description of preferred embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and similarly acting, wherein a repeated description of these elements is omitted.

1 shows a block diagram of a device 100 for influencing a text-to-speech output according to an embodiment of the present invention. The device 100 has a facility 102 to capture, a device 104 to evaluate and a device 106 to adjust to. The device 100 is in a vehicle 108 arranged. The vehicle 108 has a device 110 for outputting text as a language according to an embodiment of the present invention. The device 110 to output is adapted to perform the text-to-speech output. The device 110 to spend is with the device 100 connected to influence. Furthermore, the device 110 with a speaker 112 connected in the vehicle 108 is arranged. The device 110 to output has an interface 114 for receiving text to be output.

The device 102 for detecting in this embodiment is a camera 102 that have a detection area 116 in which objects can be mapped. The coverage area 116 is at a minimum distance from the camera 102 arranged. When an operator of the device 100 within the coverage 116 a body part 118 as a hand moves, the device detects 102 to capture the movement. When the movement of the body part 118 meets predetermined criteria, then recognizes the device 102 for detecting a gesture of the operator and provides a trajectory of movement as sign information. For example, the sign information includes from where to where the body part 118 from the operator within the detection area 116 has been moved. The device 104 to evaluate receives the sign information and evaluates it. The sign information is compared with a plurality of stored sign schemes to recognize a gesture logically linked to the gesture of the operator. The device 106 to customize receives the instruction and adjusts a parameter of the device 110 to output text as a language according to the instruction of the operator to influence the text-to-speech output.

In other words shows 1 a device 100 for a gesture-controlled voice zoom. A voice output and voice dialogue system 110 in the vehicle 108 has a TTS (Text-to-Speech) function that allows any text to be read. The use in the vehicle 108 becomes very handy and efficient with the approach presented here. For example, a read speed may be made variable, requiring a short time to get to a relevant part of the text. This allows you to capture longer text content faster, not just word by word be read out. Navigation through various elements (eg emails) and branching into the mail is possible without looking at the display for orientation and without the associated distraction. The input is primarily without display-based approaches, such as rotary trigger, touch screen or steering wheel remote control. In the speech dialogue system 110 Selection control is via gestures, which is fast and therefore efficient for longer items (eg subject lines). For a navigation in the text to be read, the gesture-controlled speech zoom is very well suited.

Free space gesture control allows intuitive and distraction-free input commands z. B. by hand. The gestures and gestures can be detected in particular e-field, radar and / or ultrasound-based.

The intuitive gestures become in the device 100 used to improve the flaws in textual navigation when using voice output while minimizing distraction. To maximize operating efficiency, several aspects can be linked. The text output itself can be controlled via the gestures, for example faster or slower and / or louder or quieter. Likewise, an output content can be controlled via the gestures. For example, you can jump from one text to another. Within a text, a text passage can be selected for output. The term "language zoom" is due to the fact that it is possible with the approach presented here, with great speed from the top level (eg newsfeeds) through the various levels (feed, article, abstract, text, paragraph, quick reading) , until the desired text passage "zoom" can.

The approach presented here can be used, for example, in head units. In principle, however, instrument clusters or center stacks as well as applications with any tablets or industrial control consoles or machines are conceivable.

2 shows a flowchart of a method 200 for influencing a text-to-speech output according to an embodiment of the present invention. The procedure 200 has a step 202 of grasping, one step 204 the evaluation and a step 206 of adapting. In step 202 Upon detecting, a gesture of an operator is detected to obtain a gesture information. In step 206 of the evaluation, the sign information is evaluated to recognize an instruction of the operator. In step 208 of matching, at least one parameter of the text-to-speech output is adjusted using the instruction to affect the text-to-speech output.

In other words shows 2 a flow diagram of an interaction concept for controlling speech output using intuitive gestures. The interaction concept combines control of the output and control of the content and, by supplementing the start and stop functions, enables complete, efficient and distraction-free navigation through text elements and text using free-space gesture control. Simple, intuitive gestures can be used. The approach presented here describes a control that manages completely without eye contact. It may be necessary to give the user orientation where he is. Likewise, it may be important to give feedback as to whether a command has been recognized. Both can be achieved together by announcing the result by voice after a gesture representing a command. For example "Subscribed Feeds" / "Article in Feed" / "read text content /" next paragraph "/" faster / slower. "In addition, there may be a voice-controlled start of the procedure 200 make sense, which can be done with a conventional voice input via commands. For example, "read SMS";"reade-mail"; Read "Newfeeds".

In the 3 . 4 . 5 and 6 is shown by way of example gestures or gestures how the navigation and voice output can be controlled.

3 shows a representation of a first gesture 300 an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention. The first gesture 300 is shown as coming from the capture device, as in 1 is shown is detected. Shown is a body part as a hand 302 within the scope 118 is moved vertically. It may be the first gesture 300 at least one upward movement and / or at least one downward movement of the hand 302 include.

Based on 3 For example, the navigation through a structure of articulated text elements will be described. To navigate through the structure (levels) become gestures 300 in vertical direction, as this is connected with higher and lower. Moving up represents a command to move to a higher structural level, a move down represents a command to move one level down. (So eg Feeds => title => paragraphs or by email (sender => subject => paragraphs)

4 shows a representation of a second gesture 400 an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention. The representation in 4 corresponds to the illustration in 3 , The hand 302 will be within the coverage 118 horizontally, transversely to a detection direction of the detection device moves. It can be the second gesture 400 at least one forward movement and / or at least one movement of the hand 302 include.

Based on 4 By way of example, the navigation through the elements of a plane is described. It becomes a vertical gesture 400 used. This left or right movement can be used analogously to a swipe on a touch screen to scroll or to select the next entries.

5 shows a representation of a third gesture 500 an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention. The representation in 5 corresponds to the representation in the 3 and 4 , The hand 302 will be within the coverage 118 circular, moved transversely to the detection direction of the detection device. This can be the third gesture 500 at least one forward movement and / or at least one movement of the hand 302 include.

Based on 5 For example, the navigation within a text is described. Intuitively, a circular movement appears here 500 , This corresponds to a control movement of a rotary knob, such as a jog shuttle. Analog can be switched here by circles from one playback speed to the next.

6 shows a representation of a fourth gesture 600 an operator of a device for influencing a text-to-speech output according to an embodiment of the present invention. In contrast to the 3 . 4 and 5 is the fourth gesture 600 transverse to the detection direction of the detection device 102 shown. The hand 302 will be within the coverage 118 on the detection device 102 to and / or from the detection device 102 moved away. The gesture 600 can be executed horizontally, for example. It can be the fourth gesture 600 at least one forward movement and / or at least one return movement of the hand 302 include.

Based on 6 For example, start / stop of the output will be described. To stop the voice playback can be a push gesture 600 (in a z-axis pointing into the plane of the drawing). To resume or start playback, a pull gesture 600 (in a Z-axis pointing out of the drawing plane). In particular, the stop gesture 600 is a natural gesture that is also used in human communication.

7 shows a representation of different hierarchy levels 700 . 702 . 704 a text-to-speech output text according to an embodiment of the present invention. The hierarchy levels 700 . 702 . 704 are shown in a table and shown for three different text types, for example. The hierarchy levels 700 . 702 . 704 are in a first level 700 , a second level 702 and a third level 704 assigned. The hierarchy levels 700 . 702 . 704 are shown as rows of the table. In a first column of the table are the hierarchy levels 700 . 702 . 704 for texts, such as message texts 706 (Newsfeed) offered. The first level 700 the message texts 706 is as a topic or resort 708 (Feed name). The second level 702 the message texts 706 is as a title and / or summary 710 (Title + Abstract). The third level 704 the message texts 706 is as a paragraph in the text 712 specified. In a second column of the table are the hierarchy levels 700 . 702 . 704 for electronic mail or e-mail 714 plotted. The first level 700 the e-mail 714 is the sender 716 specified. The second level 702 the e-mail 714 is as the subject 718 specified. The third level 704 the e-mail 714 st as a paragraph in the text 720 specified. In a third column of the table are the hierarchy levels 700 . 702 . 704 for short messages or sms 722 plotted. The first level 700 the SMS 722 is the sender 724 specified. The second level 702 the SMS 722 is as a sentence in the text 726 specified. At the SMS 722 is the third level 704 not used. In the 7 exemplified hierarchy levels 700 . 702 . 704 can be selected by the operator to read aloud. Between the individual text categories 706 . 714 . 722 can also be changed by sign. For example, the change can be made analogously to a sequential menu.

In other words shows 7 an example of navigation through text elements by structure. The outline can be used for faster navigation through larger texts in speech output. Due to the structure, text elements and texts can be quickly navigated to the relevant text passage. It is suggested to structure the text elements in order to move quickly through gestures in the structure. 3 levels 700 . 702 . 704 for the structure, for example, e-mails and message texts are sufficient.

8th shows a representation of various actions 800 for influencing a text-to-speech output according to an embodiment of the present invention. In a first column of a table are the actions 800 listed. In a second column are descriptions 802 by gestures or gestures. illustrations 804 Signs are in a third column. In a fourth column, confirmations associated with gestures or gestures to be output via the text-to-speech output are exemplary 806 plotted. The illustrations 804 are presented as a means of detecting how they are in 1 shown is the gesture or gesture captured. That's why the illustrations are 804 mirror image of the description 802 the gestures or gestures in the second column.

In a first line 808 the table is the first action of the actions 800 faster reading registered. The faster reading takes place from command to command in an increasing sequence, from normal to faster to very fast to scan. The text is read out with an assigned output speed. The description 802 the gesture to the first action is circles clockwise, next level after about 0.8 sec. As illustration 804 is a circular movement of a body part, as in 5 shown. For the second action, a circular movement of a hand counterclockwise is shown. As confirmation text 806 is offered "faster". The faster reading can be used to navigate faster through larger texts in speech output.

Usually TTS has a read speed that can be described as rather slow. This is for the good understanding of z. For example, menu items recommended in this example on the other side does not require too much time to listen. For longer texts (eg e-mails or newsfeeds), however, a faster and / or variable reading speed is advantageous. For example, three speed levels can be introduced for the voice output as well as a fourth level, a scan mode. The Normal Speed indicates a normal (moderate) reading speed. Fast reading means double speed output. The very fast reading means an output with three to four times the speed (depending on the clarity of the language engine). In the scan mode (jump) only the first five words of a sentence can be read quickly (double speed). Furthermore, only the first 3 sentences of a paragraph can be read. Then you can jump to the next paragraph. Again, gestures can be used to switch between levels.

In a second line 810 the table is the second action of the actions 800 submitted slowly reading. Slow reading is done in a descending sequence from scanning over very fast to fast to normal to stop. The description 802 to the second action is circles counterclockwise, next stage after about 0.8 sec. As illustration 804 is a circular movement of a body part, as in 5 shown. For the second action, a circular movement of a hand is shown in a clockwise direction. As confirmation text 806 is submitted "slower".

In a third line 812 the table is the third action of the actions 800 stop reading submitted. The description 802 to the third action is Z-axis in the direction of surface (push). As illustration 804 is a linear movement of a body part, as in 6 shown. For the third action, a straight-line movement of a hand to the means for detecting is shown. As confirmation text 806 is paused.

In a fourth line 814 the table is the fourth action of the actions 800 Start reading Posted. The description 802 for the fourth action, Z-axis is off the surface (pull). As illustration 804 is a linear movement of a body part, as in 6 shown. For the fourth action, a rectilinear motion of a hand away from the detection device is shown. As confirmation text 806 No command is requested. Instead, the text is read at normal speed (continued).

In a fifth line 816 the table is the fifth action of the actions 800 a change to another text element at the same hierarchical level as a currently read text element. In this case, the other text element is arranged in an order before the current text element (an element back in the same plane). The description 802 to the fifth action is horizontal movement left. As illustration 804 is a linear movement of a body part, as in 4 shown. To the fifth action is a rectilinear motion of a hand is shown from right to left. As confirmation text 806 is "back" advertised.

In a sixth line 818 the table is called the sixth action of the actions 800 a change to another text element at the same hierarchical level as a currently read text element. The other text element is arranged in order of the current text element (jump to the next element of the same level). The description 802 to the sixth action is horizontal movement right. As illustration 804 is a linear movement of a body part, as in 4 shown. For the sixth action, a straight-line movement of a hand is shown from left to right. As confirmation text 806 is offered "before".

In a seventh line 820 the table is the seventh action of the actions 800 a change to another text element at a lower hierarchical level. The other text element has a lower hierarchy level compared to the current text element (one level lower). The description 802 the seventh action is vertical downward movement. As illustration 804 is a linear movement of a body part, as in 3 shown. For the seventh action a straightforward movement of a hand is shown from top to bottom. As confirmation text 806 is a name of the hierarchical level.

In an eighth line 822 the table is the eighth action of the actions 800 a change to another text element at a higher hierarchical level. In this case, the other text element has a higher hierarchical level (one level higher) than the current text element. The description 802 The eighth action is vertical upward movement. As illustration 804 is a linear movement of a body part, as in 3 shown. For the eighth action, a straightforward movement of a hand from bottom to top is shown. As confirmation text 806 is a name of the hierarchical level.

In other words shows 8th an interaction concept for controlling speech output using intuitive gestures. Shown is an overview of the free space gesture control. Through the gestures, the speech output can be controlled free of optical distractions. The gesture control allows the navigation through text elements with structure using intuitive and thus easy to learn gestures. The approach presented here makes it possible to efficiently read out e-mails or extensive news feeds as well as text messages while driving through TTS (Text-to-Speech) due to the improved navigation through the elements and texts in the vehicle.

With the approach presented here, for example, using eight simple, intuitive gestures, a comprehensive (complete) control of a speech output by means of structured text elements and within text is possible, whereby the efficiency of the speech output is considerably increased. This is the first time a manageable use of speech output for applications such as e-mail, newsfeed, but also a significant improvement for SMS possible.

The embodiments described and shown in the figures are chosen only by way of example. Different embodiments may be combined with each other in their entirety or in relation to individual features or also multimodally with other input / output channels such as display display or voice or rotary / push-button inputs. Also, an embodiment can be supplemented by features of another embodiment.

Furthermore, method steps according to the invention can be repeated as well as carried out in a sequence other than that described.

If an exemplary embodiment comprises a "and / or" link between a first feature and a second feature, then this is to be read so that the embodiment according to one embodiment, both the first feature and the second feature and according to another embodiment either only first feature or only the second feature.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102008051757 A1 [0002]

Claims (11)

Procedure ( 200 ) for influencing a text-to-speech output to an operator, the method ( 200 ) has the following steps: capture ( 202 ) of a gesture ( 300 . 400 . 500 . 600 ) of the operator in a gesture information; Evaluate ( 204 ) the gesture information to recognize an instruction of the operator; To adjust ( 206 ) of at least one parameter of the text-to-speech output using the instruction to affect the text-to-speech output. Procedure ( 200 ) according to claim 1, wherein in step ( 202 ) grasping the gesture ( 300 . 400 . 500 . 600 ) as a spatial sequence of movements of at least one body part ( 118 ) of the operator within a detection room ( 116 ) a detection device ( 102 ) is detected. Procedure ( 200 ) according to one of the preceding claims, wherein in step ( 204 ) evaluating the sign information and the voice command are evaluated in response to a voice command of the operator to recognize the instruction of the operator. Procedure ( 200 ) according to one of the preceding claims, comprising a step of comparing the instruction with a previously recognized instruction of the operator to detect a sequence of instructions, wherein in step ( 206 ) of adjusting the parameters is further adjusted using the sequence of instructions. Procedure ( 200 ) according to one of the preceding claims, wherein in step ( 204 ) evaluating the instruction as an acceleration command, a deceleration command, a start command, a pause command, a forward command, a backward command, an up command, and / or a down command is detected. Procedure ( 200 ) according to claim 5, wherein in step ( 206 ), in response to the deceleration command, the text is output more slowly, the text-to-speech output is started in response to the start command, the text-to-speech output is triggered in response to the pause command. Output is issued in response to the forward command, a subsequent text element is output, a previous text element is output in response to the backward command, a text element of a higher hierarchical level is output in response to the up command, and / or a text element of a lower hierarchy level is output in response to the down command. Procedure ( 200 ) according to one of the preceding claims, comprising a step of issuing an acknowledgment of the recognition of the instruction. Procedure ( 200 ) according to one of the preceding claims, comprising a step of providing a name of a hierarchical level of a currently selected text segment, the name being provided in response to a beginning of the current text segment. A method of outputting text as speech, the method comprising the step of: providing a speech signal using at least one parameter obtained by a method ( 200 ) is affected according to one of claims 1 to 8. An apparatus comprising units adapted to perform or drive the steps of a method according to any one of claims 1 to 9. Computer program product with program code for carrying out the method according to one of claims 1 to 9, when the program product is executed on a device.

Images