DE10064543B4 - Method for controlling a multimedia system - Google Patents

Abstract

method for controlling a multimedia system (2) for the visualization of itself changing over time Images caused by temporally changing image information (5) in which the acoustic input of terms over at least a microphone (9) and a speech recognition system (10) takes place and in the case of a match of the acoustically entered term with a control of Index data causes a function associated with this control to characterized in that the index data from a synchronous to the image information (5) changing index information (6) are gained or derived, which change over time and the for the current image information is characteristic.

Description

The The invention relates to a method for controlling a multimedia system for the Visualization of temporally changing pictures according to the generic term Claim 1.

From the DE 195 20 586 A1 For example, an interactive game system is known in which a toy is equipped with a voice recognition module that responds to a human voice and signals from a television so that the toy can be included in the storyline of a television game series. For this purpose, additional data for the toy are transmitted in addition to the television signal in the transmission. The data is transmitted from a transmitter associated with the television, for example via a radio or infrared link to the toy. The data may be, for example, keywords, ie words on which the system is trained and responsive via speech recognition, response sentences, ie keyword responsive speech outputs or control bits. To control the toy, at least one keyword and an associated response sentence are transmitted to the toy via the radio or infrared link. The transmitted keyword is output as part of the action series shown on the TV via the television speaker and after receipt of the keyword is played by the toy of the keyword associated with the response sentence. In this case, the individual dialogues and thus also the keywords and associated answer sentences are generated before the transmission is created and these are broadcast synchronously to the transmission. In the context of the program, key words for controlling the acoustic output of the character are thus output via the television at fixed times.

The US 4,333,152 discloses a video entertainment system in which one or more viewers, by means of predetermined voice commands, can influence the course of a recorded film, to locations fixed in the course of the film. At the aforementioned selection times, the viewer can select one of several options by entering a predefined keyword.

task The invention is to show a method which it with a relatively low cost, an integration of connections or hyperlinks in temporally changing or moving pictures allows.

to solution This object is a method according to the claim 1 formed.

Of the essential aspect of the method according to the invention is to see that the index data changes from a synchronous to the image information Index Information be gained or derived, which change over time and the for the each current image information is characteristic. Advantageous in the invention to the actual image information, from the visual representation of the temporally changing images takes place, a synchronous Generates index information that contains certain controls that at agreement with a term entered acoustically by the user Trigger function, for example, the connection or link to other information, Images, etc. of system internal or external sources of information and the visualization of this information, pictures etc ..

Of the The invention is based on the recognition that referencing to objects or elements in moving pictures is a considerable effort, since it would be necessary in every single frame of the image information the respective link mark it with a pointer instrument (eg mouse) to be able to refer. Further would be it also for the user extremely difficult to refer to objects that move quickly and / or small are shown and / or disappear behind other objects. With the method according to the invention such problems are avoided.

further developments The invention are the subject of the dependent claims. The invention is in Explained below with reference to the figures of exemplary embodiments. It demonstrate:

1 in a schematic functional representation of a first possible embodiment of a system for carrying out the method according to the invention;

2 a schematic representation of an image track consisting of a plurality of temporally successive image frames and an index track consisting of a plurality of successive index frames, together with a time window and a time-dependent index list generated using this time window ;

3 in a simplified representation the structure of the index track;

4 another, opposite the 1 extended version of a system for carrying out the method according to the invention;

5 a system for generating the In dex-track.

In the 1 is 1 a user and 2 a multimedia system with an associated screen 3 in the embodiment of video movie playback described below by way of example. The required data are in the multimedia system 2 as digital audio information or audio track 4 with a plurality of temporally successive frames, as digital image information or image or video track 5 with a multitude of temporally consecutive image frames 5 ' and additionally as an index track 6 with a multitude of temporally consecutive index frames 6 ' to disposal. These index frames 6 ' include index lists of controls, link destinations or addresses, control commands to control the multimedia system, and / or extension to a knowledge base, and so on.

The tracks 4 . 5 and 6 are synchronized in time, so not just each to a specific image of the video track 5 also the correct sound information is present, but also a certain index frame 6 ' with the associated content.

The tracks 4 . 5 and 6 are available to the multimedia system on, for example, an internal video source, such as a VCR or CVC recorder, or are provided to the multimedia system via a data link, such as a satellite receiving antenna, broadband cable, etc., from the outside.

In the 2 is with 7 a time window is displayed which is over a given or preselected number of image tracks 5 ' and thus also over a given, equal number of index tracks 6 ' extends. With this time window 7 In the manner to be described in more detail below, the system becomes a time-dependent index list 8th whose contents are the contents of all index lists of the index frames 6 ' within the time window 7 equivalent.

The basic operation of the system is that the user 1 while watching the on screen 3 displayed images by voice input via a microphone 9 certain terms in a speech recognition system 10 and that then in this system the inputted term is converted into input information and compared with timed index list controls or an extended index list created from this time dependent index list, where the input information matches with a control then the action associated with that control is triggered, e.g. For example, adding additional information to the screen 3 , eg with simultaneous interruption of video playback.

In the in the 3 For the example of the index track structure, the time-dependent index list contains the term "Derrick" as well as the term "man" as a control. Both terms are by within the time window 7 reproduced image sequences for the user, as these image sequences Derrick or a man in the picture show. So is the user 1 over the microphone 9 within the relevant time window 7 Entered the term "Derrick" or "man", so be on the screen 3 Further information on "Derrick" or a list to select more information on the term "Derrick" appears, which in turn the user through voice input through the microphone 9 from the displayed list. The further information will be through the multimedia system 2 for example, retrieved from internal or external data sources.

In detail, the system concept for carrying out the method according to the invention additionally has a system 11 on which in the 1 is called "Index Track - Extraction" and which one from the index track 6 using the time window 7 the time-dependent index list 8th forms. Among other things, this will become a knowledge-based system 12 (KB system), which, using the content of the time-dependent index list 8th and, using a knowledge-based system based, for example, on a semantic network, an extended index list 13 forms. This knowledge-based system contains z. B. Dependencies between different terms, such as Object>Person>Man>Inspector> Derrick, that is, if that of the user 1 over the microphone 9 entered term during the considered time window 7 "Person", "Man" or "Inspector" is triggered, the action associated with the term or control "Derrick". Because the extended index list 13 on the time-dependent index list 8th based, is ultimately the extended index list 13 a time-dependent index list.

The extended index list 13 So it is fed to the speech recognition system. Corresponds to the over the microphone 9 entered term a control of this extended index list 13 so causes the processing 14 the implementation of the respective control associated action in the multimedia system 2 ,

The further content of the time-dependent index list is in the illustrated embodiment of the device 11 as index and action list 15 also directly to the processing 14 available so that these then match the input via the microphone 9 also with a control of the extended index list 13 can initiate the appropriate action.

The 4 shows as a further possible embodiment, a system concept according to the invention, which instead of processing 14 a so-called retrieval machine 16 contains and the system 11 for the extraction of the index track a database 17 Among other things, a ring buffer function for the caching of a larger number of index frames 6 ' has and also the time-dependent index list 7 forms, over an adjustable time window 7 , Will not match the input via the microphone 9 with a control of the extended index list 13 determined, so causes the retrieval machine 16 a change of the time window 7 , For example, a shift and / or enlargement of this time window, or via the KB system 12 the creation of an extended index list 13 on a larger scale.

Basically, these changes of the time window 7 and / or the extended index list 13 if the input does not match via the microphone 9 with a control of the extended index list 13 can also be made in several stages and / or it is possible that initially the KB system provides an index list, the content of which is equal to the time window 7 certain time-dependent index list 8th is and that the creation of an extended index list by the KB system 12 is caused only if in a previous attempt no match of the input via the microphone 9 with a control of the extended index list 13 was determined.

• 1 Benutzer
• 2 Multimedia-System
• 3 Bildschirm
• 9 Mikrophon
• 10 Spracherkennungssystem.

For completeness, the shows 5 another system concept for creating the index information or the index track. In the 5 the following assignments apply:

• 1 user
• 2 Multimedia system
• 3 screen
• 9 microphone
• 10 Speech recognition system.

The speech recognition system is a lexicon or memory 18 (Thesaurus) associated with a variety of the system understandable and thus permissible terms or inputs.

The creation of the index track 6 is done by the index track designer or user 1 when looking at the screen 3 shown image sequence, for example, the video film on the microphone 9 , being to the over the microphone 9 entered terms, for example, the commands or actions assigned in these terms via the microphone 9 be entered.

As far as these voice inputs with terms in memory 18 match, these or the corresponding inputs to the processing 19 then passed the index track generation 20 to generate the index track 6 or the sequence of temporally consecutive index frames 6 ' causes.

The The invention has been described above with reference to exemplary embodiments. It is understood that many changes as well as modifications possible are without it the inventive idea underlying the invention is abandoned. So it is possible in particular the above-described system or method not only for the creation of shortcuts links between a video and other information or data sources available standing information, etc., but the inventive method is generally time-sensitive Representations or pictures applicable.

1

user

2

Multimedia system

3

screen

4

sound information or audio track

5

digital Picture information or video track

6

digital Index information or index track

4 '

Audio Track Frame

5 '

Video track frame

6 '

Index-track frame

7

Time window

8th

time-dependent index list

9

microphone

10

Voice recognition system

11

Index-track extraction

12

Survey System

13

Advanced Index or term list

14

processing

15

Index- and action list

16

Retrieval Machine

17

database

18

Storage or thesaurus

19

processing

20

Index-track production

Claims (9)

Method for controlling a multimedia system ( 2 ) for the visualization of temporally changing images caused by temporally changing image information ( 5 ), in which the acoustic input of terms via at least one microphone ( 9 ) and a voice recognition system ( 10 ) and in which a match of the acoustically inputted term with a control of index data causes a function associated with this control, characterized in that the index data from a synchronous to the image information ( 5 ) changing index information ( 6 ) are obtained or derived, which change with time and which are characteristic of the current image information. Method according to Claim 1, characterized in that the image information ( 5 ) as well as the index information ( 6 ) consists of several temporally successive sequences or frames. Method according to Claim 2, characterized in that by means of a time window ( 7 ) whose length is a plurality of frames of image information ( 5 ) and the index information ( 6 ), an index list ( 8th ), which controls all controls of all index frames ( 6 ' ) within the time window ( 7 ), and that this time-dependent index list is used as index data for the speech recognition system ( 10 ) is used. Method according to Claim 2 or 3, characterized in that by means of a time window ( 7 ) whose length is a plurality of frames of image information ( 5 ) and the index information corresponds to an index list ( 8th ), which controls all controls of all index frames ( 6 ' ) within the time window ( 7 ) and that from this time-dependent index list the index data for the speech recognition system ( 10 ) to be generated. Method according to claim 4, characterized in that from the time window ( 7 ) obtained time-dependent index list by expanding the index data ( 13 ) be won. Method according to one of the preceding claims, characterized in that, if the input does not match an element of the index data, a change of the time window ( 8th ), for example, an enlargement of the time window takes place. Method according to one of the preceding claims, characterized in that the control elements of the index information are each characteristic of elements of the image information ( 5 ) are. Method according to one of the preceding claims, characterized characterized in that triggered in the multimedia system Function a link or link function, for example, for visualization of other image sequences or information. Method according to one of the preceding claims, characterized in that for generating the index information via at least one microphone ( 9 ) and a speech recognition system ( 10 ) are input acoustically to the image content of the visually visibly displayed, temporally changing images, the speech recognition system ( 10 ) forms from these acoustic terms respectively corresponding controls that are stored in sync with the changing images as index information.