JP3225428B2 - Multimedia information providing method and multimedia information system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for providing multimedia information and a multimedia information system, and more particularly to a method for editing, managing and presenting multimedia data such as video, audio and text to a user. The present invention relates to a multimedia information providing method and a multimedia information system for switching multimedia data presented based on a user's interactive operation.

[0002]

2. Description of the Related Art In a multimedia information system for effectively presenting to users by variously combining multimedia data such as video, audio, and text,
A system that can easily describe a combination of multimedia data is desired.

[0003] For this reason, a method of describing a temporal timing or the like at which multimedia data is presented as a scenario is disclosed, for example, in Japanese Patent Application No. 7-107415 "Multimedia Data Management Method and Apparatus". In this example, the presentation timing of the multimedia data and the presentation effect indicating the screen position at the time of presentation are described as a scenario, and the scenario can be described hierarchically. Scenarios can be described in units of semantic units.

[0004] As a result, it is easy to edit and search for a scenario, and it is possible to create a new scenario by combining already created scenarios, so that the scenario can be easily distributed and reused as a part. The work efficiency in multimedia application production has been improved.

As described above, in the conventional multimedia data management method and apparatus using a scenario, a scenario can be described based on a scenario editor's intention for the purpose of improving the work efficiency of the scenario editor. By combining existing scenarios, larger scenarios can be easily realized.

[0006]

However, the conventional multimedia data management method and apparatus have the following problems. First, since the scenario to be described was created based solely on the intention of the scenario editor, the user, that is, the person receiving the multimedia data using the created scenario, presented the multimedia data. In the process of receiving, there is a problem that it is not possible to receive an interactive presentation that actively participates in changing the subsequent presentation, or that the scenario editor cannot write a scenario that enables interactive presentation .

Further, applications such as presentation and switching of multimedia data based on a user's instruction, such as a “travel guide system” using multimedia,
In order to realize “English conversation teaching materials” and the like, it is necessary to realize a switching control mechanism in any program that describes a multimedia application. Therefore, in order to improve the interactivity, the input method of the switching instruction is improved, the method of presenting the input guide for facilitating the input of the switching instruction by the user, or the timing of presenting the guide is changed. There is a problem that a program describing an application must be changed every time a part or all of the multimedia data is changed.

[0008] The present invention has been made in view of the above points, and solves the above-mentioned conventional problems. The present invention can describe, as a scenario, an application of switching multimedia data to be presented interactively based on a user's instruction as a scenario. An object of the present invention is to provide a media information providing method and a multimedia information system.

It is a further object of the present invention to provide a multimedia information providing method and a multimedia information system capable of realizing the above-mentioned application in which the user interactively switches scenarios without changing the multimedia application. That is.

[0010]

SUMMARY OF THE INVENTION A multimedia information providing method of the present invention presents to a user multimedia information including video and audio, still images, texts, or documents requiring temporally continuous reproduction. At this time, the presentation method is managed as a scenario, and the user is urged to instruct a branch based on the branch condition described in the scenario.

FIG. 1 is a diagram for explaining the principle of the present invention. The multimedia information providing method according to the present invention includes a video and an audio, a still image,
At least one of multimedia information including text or document is stored in the first storage unit as a material (step 1), and a material that requires temporally continuous playback of the stored material is required. As for, a time fragment of the material is used as a unit, and for a material that does not need to be reproduced continuously in time, a time interval for continuing the reproduction is given as a unit (step 2), and synchronization or The selection relation for selecting and reproducing one of a plurality of subsequent segments based on the continuous temporal connection relation and the user's instruction is described as a scenario, and stored in the second storage means. Every (step 3), any one of the scenarios stored in the second storage means is read, and a desired material is stored in the first storage means based on the relationship between the segments described in the read scenario. Read and reproduce (step 4). In the course of the reproduction, if it becomes necessary to branch the scenario based on the selection relation, the user can determine which of the branch destinations based on the selection relation described in the scenario. A guide for enabling the selection is presented to the user before the time of branching (step 5).

In step 5, when the guide is presented to the user, a plurality of types of guides are created and stored in advance, and a guide suitable for the user to instruct the branch of the scenario is stored. The selected guide is selected from a plurality of guides, and the selected guide is presented to the user.

In step 5, when presenting a guide to the user, the user inputs a selection instruction to the system after presenting the guide and the complexity of the branch described in the stored scenario. Based on at least one of the response time histories or both, a time at which a guide for branching is presented to the user is determined.

In step 5, when the guide is presented to the user, the guide is presented based on a fixed time for presenting the guide. FIG. 2 is a diagram illustrating the principle of the present invention. The multimedia information system of the present invention can be used for video and audio that require continuous playback in time,
A material storage unit 110 that stores at least one of multimedia information including a still image, a text, or a document as a material, and temporally continuous playback of the material stored in the material storage unit 110 is required. For a material, a time fragment of the material is used as a fragment, and for a material that does not need to be reproduced continuously in time, a time interval for continuing reproduction is given as a fragment, and synchronization or continuous time is used between the fragments. Storage unit 130 that describes, as a scenario, a typical connection relationship and a selection relationship for selecting and reproducing any of a plurality of subsequent segments based on a user's instruction, and stores the described scenario. And reads out any one of the scenarios stored in the scenario storage means 130 and, based on the relation between the segments described in the read out scenario, a desired material. A material reproducing means 120 for reading and reproducing from the material storage stage 110, and in the material reproducing means 120, when a scenario needs to be branched based on the selection relation at the time of generating the scenario, the selection relation described in the scenario is changed. Based on the user, select one of the branch destinations
Create and save multiple types of guides in advance to make it possible
Guide storage means to be performed and the user instructs branching of scenarios
Suitable guides to be saved in the guide storage means
A guide selecting means for selecting from among a plurality of guides, Sina
Minutes described in the scenario stored in the Rio storage
The degree of complexity and the user's choice
Response time history before inputting indications to the system
And / or both based on
Guide presentation time control hand that determines the guide presentation time
The time at which branching is performed by the stage and
Guide presenting means 160 for presenting a guide to the user before
And

[0015]

[0016]

As described above, the multimedia information system of the present invention can easily realize a multimedia information system capable of switching the flow of a presentation according to a user's interactive selection. The flow of the presentation is stored and managed in scenario storage means created independently of the application program as a scenario. The guide presenting unit presents a guide to the user to request the user to input a branch instruction based on the branch condition described in the scenario. As described above, the application program does not need to know what branch is described in the scenario, and the creator can create the application program regardless of the content of the presentation.

Therefore, the scenario editor who creates the contents of the presentation or the flow of the presentation can create, describe, and change the flow of the presentation by editing the scenario of the presentation. That is, the presentation can be changed without changing the application program.

Furthermore, in order to make an interactive presentation, it is easy to change the guide to be presented to the user and to change the presentation timing.
A plurality of guides created in advance are stored in the guide storage means, and appropriate guides are associated with the branching conditions described in the scenario, and when a branch is required in the course of the reproduction based on the scenario. And a guide suitable for the branch condition there. Under which branch condition, which guide is presented can be associated irrespective of the application program, so that a multimedia information system for performing an interactive presentation without changing the application program can be realized.

The guide presentation time control means can determine how long a guide should be presented at a scenario branch point, that is, a time at which a branch instruction is required. Determine the guide presentation time at the next branch without changing the application program, depending on the complexity of the branch described in the scenario or the history of the time until the user recognizes and reacts to the guide. Can be.

Further, by fixedly setting the time at which a branch instruction is required, the configuration of the system becomes easy.

[0022]

FIG. 3 shows the configuration of a multimedia information system according to the present invention. In the figure, a multimedia information system 100 includes a material storage unit 110, a segment playback unit 120, a scenario storage unit 130, a scenario analysis unit 1
40, guide holding unit 150, guide presenting unit 160, user request input unit 170, and guide presenting time control unit 180
It is composed of

The material storage section 110 is a mechanism capable of storing and managing multimedia information such as video and audio data and reading out the data in units of a unit. Usually, each material is digitally encoded and stored in a magnetic disk drive. It is realized by a method such as storing. Here, a segment refers to a temporal fragment of a material such as a video and an audio that requires temporally continuous reproduction, and a temporally continuous reproduction of a still image, a text, a document, or the like. For those materials that do not need to be reproduced, a time interval for continuing the reproduction is given.

Upon receiving an instruction from a scenario interpreting unit 140, which will be described later, the segment reproducing unit 120 reads out the material from the material storing unit 110 in units of a unit, and reproduces the material by a reproducing method suitable for each material. Present to the user. As described above, since the segment is explicitly or implicitly provided with an attribute relating to the time required to reproduce the segment, the segment is reproduced in accordance with the order of the segments passed from the scenario interpretation unit 140. Can be continued.

The scenario storage unit 130 is a mechanism for accumulating and managing a scenario in which the contents of a presentation or the flow of a presentation are described in units of units. The scenario is stored as a sequence of character codes or as a result of compilation. And can be read out in units of scenarios.

The scenario interpreting section 140 is a mechanism for reading the scenario stored in the scenario storing section 130 and developing the read scenario as a sequence of segments on the time axis. Even in the case of a scenario described hierarchically including sub-scenarios, the scenario necessary for deployment is read out from the scenario storage unit 130 and deployed repeatedly as necessary, so that the temporal You can get a line. If there is a recursive call of the scenario, it is possible that the scenario cannot be expanded. However, since it is not the issue of the present invention, here, only the scenario that can be finally expanded will be described.

The guide storage unit 150 is a mechanism that stores a plurality of guides suitable for allowing a user to select a branch described in a scenario, and can read out each guide as required. The guide presenting unit 160 is a mechanism for reading out an appropriate guide from the guide holding unit 150 based on the branch state of the scenario analyzed by the scenario analyzing unit 140 and presenting it to the user.

The user request input unit 170 is a mechanism for the user to input information for selecting and instructing the branching direction of the scenario based on the guide presented by the guide presenting unit 160. This can be realized by a method such as a user clicking the guide presented on the screen with a mouse.

The guide presenting time control unit 180 controls how long the guide presented by the guide presenting unit 160 based on the branch of the scenario is presented before the time at which the branch is performed. It is. In general, when branching is complicated, it is considered that a user needs a lot of time to select which branching method. Therefore, it is necessary to make the time for presenting the guide in advance relatively long. It is considered effective in improving convenience. On the other hand, in the case where an instruction to select a branch direction is given after a certain time has elapsed since the guide was presented as a characteristic of each user, it is more convenient to present the guide closer to the branch point. For this purpose, as shown in FIG. 4, the guide presentation time control unit 180 accumulates the user's past operation history, that is, the history of the time from the presentation of the guide to the instruction to select a branch. This is realized by including the instruction history information storage unit 182.

[0030]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below using specific scenarios. FIG. 5 is a diagram (part 1) illustrating an example of a scenario according to an embodiment of the present invention. FIG. 6 shows a configuration of a scenario including branching in two directions, and FIG. 6 is an example of video display based on the scenario shown in FIG.

In the example of the scenario shown in FIG.
Is the time required for reproduction, and when reproduction starts at time t = 0, the reproduction ends at time t = T. The scenario is described as a scenario such that, when the reproduction of the video element A is completed, either the video element B or the video element C is selectively reproduced. The branch condition of which of the video unit B and the video unit C is reproduced is a time t
If the user instructs "turn right" before T = T, the video unit B is reproduced, and if the user instructs "turn left", the video unit C is reproduced.

The above operation will be described in more detail using the display example of FIG. FIG. 6 illustrates a display state of the video element A at time t = T−Δt.
Guides for “turn right” and “turn left” are displayed. It should be noted that the guides for "turn right" and "turn left" that are displayed on the screen are not photographed in the video unit A but are displayed by the function of the guide presenting unit 160. That is.

During the remaining reproduction time Δt of the video element A, the person riding the bicycle reaches the three-forked road, and the user must be in the “mountain direction” or within the time Δt. , "Lake direction" must be selected. Here, the video that is played when “mountain direction” is selected is
The video element B shown in the example of the scenario in FIG. 5 and the video reproduced when “lake direction” is selected is the video element C shown in the example of the scenario in FIG.

Therefore, at time t = T, the segment reproducing unit 120, following the reproduction of the image segment A,
Alternatively, one of the video segments C is reproduced based on the user's selection. The flow of control for the user to select a branch described in such a scenario and perform interactive reproduction will be described with reference to FIG. FIG. 7 is a flowchart showing a series of operations of one embodiment of the present invention.

Step 101) Scenario interpreter 140
Reads the scenario shown in FIG. 5 from the scenario storage unit 130 based on an instruction from a control unit (not shown),
At first (time t = 0), the reproduction of the video unit A is instructed to the unit reproducing unit 120. The segment playback unit 120 reads out the video segment A from the material storage unit 110 and starts playback thereof.

Step 102) Scenario interpreter 140
Interprets the scenario, recognizes that there is a branch at time T, and obtains the branch condition at that time. The obtained branch condition is passed to the guide presentation unit 160 and the guide presentation time control unit 180.
In the example of FIG. 5, the branch condition is “turn right” or “turn left”.
It is.

Step 103) The guide presenting section 160 determines a guide suitable for the branch condition received from the scenario interpreting section 140, and acquires a guide to be presented from the guide holding section 150. In this example, "right turn" and "left turn" shown in FIG. Step 104) The guide presenting time control unit 180 determines the guide presenting time t = T−Δt in consideration of the complexity of the branch received from the scenario interpreting unit 140, values of past user response times, and the like. . This value is displayed in the guide presentation unit 16
Pass to 0.

Step 105) When the playback of the video segment A by the segment playback unit 120 proceeds, and when the time t = T−Δt is reached, the guide presenting unit 160 sets the above guide, that is, “turn right” and “turn left”. Is displayed on the screen. Although the position to be presented is not mentioned, it may be a part of the image display screen as shown in FIG. 6 or an appropriate position close to the image display screen.

Step 106) The user selects one of the guides presented by the guide presenting unit 160, and notifies the system of the result. For this reason, the mechanism for receiving the user's instruction is the user request input unit 170. For example, when the user indicates one of the presented guides “turn right” and “turn left” using a pointing device such as a mouse, the user request input unit 170 determines which guide is indicated from the coordinates. I do.

Step 107) The segment reproducing unit 120
When the time t = T is reached, either the video unit B or the video unit C is reproduced based on the branch specified by the user. At time t = T, in order to start reproduction of the video element B or C without delay, part or all of both video elements are read from the material storage unit 110 in advance and prepared for reproduction. Is also possible.

Step 108) Up to step 107, the branch at time t = T is completed. here,
When the next branch is reached, the process is repeated from step 102 described above. Step 109) The reproduction of the video is continued until the next branch is reached.

As described above, the above procedure is repeated each time a branch is reached, thereby realizing a branch based on a user's interactive instruction. FIG. 8 is a diagram (part 2) illustrating an example of a scenario according to an embodiment of the present invention. The figure shows
FIG. 9 shows an example of a scenario including branching in three directions, and FIG. 9 shows a display example of an image in this case. In this example, the directions of branching at time t = T are three directions, and as shown in FIG. 9, three guides of “left turn”, “straight ahead”, and “right turn” are provided.
The type is presented.

In this example, since the direction of the branch is increased from two directions in the example of FIG. 5 to three directions, it is considered that the user needs more time to select the branch, and the guide presentation time is considered. In the control unit 180, Δt can be set large. The instruction history information storage unit 1 shown in FIG.
The average time until the user's instruction stored in 82,
The maximum time or a time in consideration of the response characteristics may be set as Δt.

As described above, in the embodiment of the present invention, the case where the guide presentation time is changed by the guide presentation time control unit 180 based on the complexity of the branch, the response characteristics of the user, and the like has been described. It is also possible to provide a simple time by giving an appropriate time in advance and presenting a guide only at the time interval. In this case, the guide presentation time control unit 180 is unnecessary, and the realization is easier.

The guide presenting unit 160 reads out an appropriate guide from the guide storing unit 150 and presents it to the user. However, the guide presenting unit 160 generates a guide based on some rule. Thus, the guide storage unit 150 can be omitted. For example, a pattern in which a character string of the branch condition “left turn” or “right turn” described in the scenario is appropriately modified and presented may be considered. As in the example of FIG. 6, a character string may be embedded in an ellipse colored with a guide. Similarly, in the example of FIG. 9, three types of character strings, “left turn”, “straight ahead”, and “right turn”, which are branch conditions, can be expressed by being embedded in a colored ellipse.

With such an expression, the guide storage unit 1
Even a scenario including a branch where there is no appropriate guide stored in the guide 50 can be easily handled. For example, in the example of FIG. 9, even when the guide to be presented is “mountain direction”, “town direction”, or “lake direction”, the character string “mountain direction”, This can be realized by assigning “town direction” and “lake direction” and using the above guide generation rule.

In the embodiment of the present invention, the example of displaying the right turn / left turn mark on the screen has been described. However, not only the right turn / left turn mark is displayed but also the display is made to blink, or the interval of the blink is set to a branch point. It is also possible to make a presentation that draws the user's attention, such as shortening it, as it gets closer. Such a presentation method can be easily realized by the guide presentation unit 160 in response to an instruction from the guide presentation time control unit 180.

In the embodiment of the present invention, the case where an image is used as a guide has been described. However, media other than the image, for example, a voice is used to guide the user to the selection, or a combination of the audio and the image is used. A multimedia information system can be realized by combining various guide presentation methods such as guidance.

The present invention is not limited to the above embodiment, but can be variously modified and applied within the scope of the claims.

[0050]

As described above, in the multimedia information providing method and the multimedia information system according to the present invention, since the branch between the segments to be reproduced can be described in the scenario, the user can interactively select the multimedia information. The system can be easily realized.

That is, the flow of the presentation is stored and managed as a scenario in a scenario storage unit created independently of the application program, and no matter what branch is described in the scenario, only one application program is used. Can handle, and create application programs regardless of the content of the presentation.

For this reason, the scenario editor who creates the contents of the presentation or the flow of the presentation creates and describes the contents and the flow of the presentation by editing the scenario of the presentation.
Can be changed. Further, the developer of the application program can develop the application program without being affected by a scenario including any branch.

Further, in order to make an interactive presentation, it is easy to change the guide to be presented to the user and to change the presentation timing.
How to present a guide that is convenient for users, that is,
Physical conditions such as the type, shape, size, etc. of the guide to be presented,
Also, it is possible to implement a multimedia information system that fully considers the time of presentation. These conditions can be easily changed even after the system or scenario is completed, and applications such as automatically changing the presentation method from the user's application history can be easily realized.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of a multimedia information system of the present invention.

FIG. 4 is a configuration diagram of a guide presentation time control unit of the present invention.

FIG. 5 is a diagram (part 1) illustrating an example of a scenario according to an embodiment of the present invention;

6 is a display example of an image based on the scenario shown in FIG. 5 in one embodiment of the present invention.

FIG. 7 is a flowchart showing a series of operations of one embodiment of the present invention.

FIG. 8 is a diagram (part 2) illustrating an example of a scenario according to an embodiment of the present invention;

9 is a display example of an image based on the scenario shown in FIG. 8 according to one embodiment of the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 multimedia information system 110 material storage unit, material storage unit 120 unit playback unit, material playback unit 130 scenario storage unit, scenario storage unit 140 scenario interpretation unit 150 guide holding unit 160 guide presentation unit, guide presentation unit 170 user request Input unit 180 Guide presentation time control unit 181 Instruction unit 182 Instruction history information storage unit

Continuation of the front page (56) References JP-A-7-141400 (JP, A) JP-A-3-9393 (JP, A) JP-A-6-105827 (JP, A) JP-A-6-60026 (JP) , A) JP-A-6-149517 (JP, A) JP-A-1-178238 (JP, A) JP-A-7-37118 (JP, A) IEICE Technical Report Vol. 92, No. 23 (1992-4-25) p.p. 29-36 (AI92-5) FUJITS U Vol. 46, No. 1 (1995-1-10) pp. 105-108 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06F 19/00 G06F 17/60 G06F 3/00 G06F 12/00 G06F 17/30 JICST file (JOIS)

Claims (5)

(57) [Claims] (1)Video requiring continuous playback in time
And audio, still images, text or documents
At least one of the multimedia information including
Stored in the first storage means, The stored material is continuously replayed in time.
For materials that require raw material, the time fragment of the material
Materials that do not require continuous playback in time
Gives a time segment to continue playback, Synchronous or continuous temporal connection between the segments
And any of multiple subsequent units based on the user's instructions
The scenario is a selection relationship for selecting and playing
And stored in the second storage means, Any of the scenarios stored in the second storage means
Read, and between the segments described in the read scenario
A desired material based on the relationship is obtained from the first storage unit.
Read, play, If it becomes necessary to branch the scenario during the playback,
Based on the selection relationships described in the scenario, the user
A guide for selecting one of the branch destinations
Characterized by being presented to the user before the time of performing
Multimedia information provision method. (2)When presenting the guide to the user
To Create and save multiple types of guides in advance, From the saved multiple guides,
Select the appropriate guide for the user to direct the branch, 2. The method according to claim 1, wherein the selected guide is presented to a user.
How to provide multimedia information. (3)When presenting the guide to the user
To The scenario described in the scenario stored in the second storage means is described.
The complexity of the branch and the user who presented the guide
The response time until the user enters a selection instruction into the system.
Use based on either or both
To determine the time to present a guide for branching to the elderly
Item 2. The multimedia information providing method according to Item 1. (4)When presenting the guide to the user
To At a fixed time set to present the guide
2. The multimedia information provider according to claim 1, wherein the multimedia information provider
Service method. (5)Video requiring continuous playback in time
And audio, still images, text or documents
At least one of the multimedia information including
Material storage means for storing; The material stored in the material storage means is temporally
For materials that require continuous playback, the time of the material
Fragments are used as fragments to produce material that does not require continuous playback in time
In addition, a time interval for continuing playback is given as a unit,
Synchronous or continuous temporal connection between the segments,
And any of multiple subsequent units based on the user's instructions
The scenario is a selection relationship for selecting and playing
Scenario scenario that stores the described scenario.
Delivery means, Any of the scenarios stored in the scenario storage means
Is read, and between the segments described in the read scenario.
The desired material is read from the material storage means based on the relationship
Material playback means to play out In the material reproducing means, when the scenario is generated,
Scenario branching is required based on selection relationships
Based on the selection relationships described in the scenario above.
Guide to allow users to select one of the branch destinations
Guide storage means for creating and storing a plurality of types in advance, An appropriate guide for the user to indicate the branch of the scenario
Of the multiple guides stored in the guide storage means.
Guide selection means to select from among, Described in the scenario stored in the scenario storage means
Use after presenting the guide and the complexity of the branch
Response time before the user enters a selection instruction into the system.
History based on either or both
Guide that determines the time to present a guide for branching
Indicating time control means, From the time of branching by the guide presentation time control means
Guide presentation means for presenting the guide to the user before
A multimedia information system comprising: