JPH09312823A - Multimedia information service method and system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily describe application such as switching, etc., of presentation effect of multimedia(MM) as a scenario based on user's conditions by reproducing material in response to a scenario whose material is the MM that temporally needs consecutive reproduction and which is generated by adding effect to in response to the reproduction. SOLUTION: One piece of MM information is stored as material. When material needs temporally consecutive reproduction, a temporal piece of the material is a segment. When material does not need temporally consecutive reproduction, a time interval which continues reproduction is added to the material and it becomes a segment. There are three types of segment relations, i.e., segments which synchronize with each other, continuously temporal connection relation and multiple segments which follow based on a user's instruction. Any of these segments is selected, and a relation to reproduce the segments is written as a scenario and stored. Any of these scenarios is read, described relation between segments is interpreted and desired material is read from stored material and reproduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia information providing method and system, and more particularly, to a method of implementing a multimedia information system, for editing, managing and user of multimedia data such as video, audio and text. The present invention relates to a multimedia information providing method and system for changing the presentation method of the above and enabling more effective information provision.

[0002]

2. Description of the Related Art In a multimedia information system for effectively presenting a user with various combinations of multimedia data such as video, audio, text, etc., a method for easily describing the combination of multimedia data is desired. .

For this reason, a method of describing the time timing of presenting multimedia data as a scenario is disclosed in, for example, Japanese Patent Application No. 7-107415 "Multimedia data management method and apparatus". In this example, the presentation timing of the multimedia data and the presentation effect for instructing the screen position at the time of presentation are described as a scenario, and the scenario can be described in a hierarchical manner, so that the scenario editor can meet the intention. The scenario can be described in units of semantic units.

As a result, the scenario can be easily edited and searched, and the already created scenarios can be combined to create a new scenario. Therefore, the scenario can be distributed and reused as a component. It becomes easier and the work efficiency in multimedia application production can be improved.

[0005]

However, the multimedia data management method and apparatus (Japanese Patent Application No. 7-107415) according to the above conventional technique have the following problems. The presentation effect description method only describes the temporal and spatial arrangement of a plurality of pieces or scenarios, and it is not possible to describe the reproduction effect of the pieces such as variable speed reproduction of video. There is a limit that it is impossible.

Therefore, the only means for realizing the above-mentioned reproduction effect is to realize variable speed reproduction on the reproduction device based on a user's instruction, and it is said that the intention of the scenario editor is sufficiently reflected. I can not say. Furthermore, in the application of switching back the presentation effect of multimedia data based on the presentation scene and the situation of the user, for example, in the case of "English conversation teaching materials" applying multimedia, the scene where the same audio material is used and the user's situation In order to realize a presentation effect such as changing the playback speed according to the learning level, it is necessary to edit the audio material itself and define it as another unit.

[0007] In addition, in a service for providing multimedia information, even in applications such as switching the quality of multimedia data presented in accordance with a fee to be paid by a user, various data can be created by editing the material data in advance. It is necessary to define them as separate pieces with various qualities.

The present invention has been made in view of the above points, and it is possible to easily describe the application of solving the above conventional problems and switching the presentation effect of multimedia data based on the situation of the user as a scenario. It is an object of the present invention to provide a multimedia information providing method and system capable of providing multimedia information.

A further object of the present invention is to realize applications such as switching the presentation effect of multimedia data based on the above scenes and user situations without changing the material or multimedia application. It is to provide a possible multimedia information providing method and system.

[0010]

FIG. 1 is a diagram for explaining the principle of the present invention. The present invention relates to a multimedia information providing method for providing multimedia information to a user, wherein multimedia information including video and audio, still image, text or document which require continuous reproduction in time is used as a material. The effect corresponding to the reproduction of is added to the segment of the scenario to generate the scenario, and the material is reproduced based on the generated scenario.

According to the present invention, at least one piece of multimedia information is stored as a material (step 1), and for the material which requires continuous reproduction in time with respect to the stored material, the material is stored. For a material that does not require continuous playback in time, using time fragments as segments, assign a time interval to continue playback as segments, and synchronize between segments, or a continuous temporal connection relationship, and A selection relation for selecting and reproducing any one of a plurality of subsequent pieces based on a user's instruction is described as a scenario and stored (step 2), and one of the stored scenarios is stored. Read out, interpret the relationship between the pieces described,
A desired material is read out from the stored material and reproduced (step 3).

In step 2, when describing a scenario to be stored, a segment or a replay effect segment describing temporal and spatial attributes when replaying a scenario is referred to as the replay effect segment or scenario. It is given as a fragment having a synchronous relationship. In step 3, when the stored material is read and reproduced, the reproduction effect described in the reproduction effect segment is interpreted as an appropriate reproduction effect attribute for each material medium, and added to the original reproduction attribute of the material. Then, the element reproduction is performed in which the desired effect described in the reproduction effect element is added.

In step 3, a scenario in which any one of a plurality of reproduction effect pieces can be selected when the piece reproduction is performed is stored, and the reproduction of the material presented according to the user's request or the situation at the time of reproduction is stored. Change the effect. FIG.
It is a principle block diagram of this invention.

According to the present invention, in a multimedia information providing system for providing multimedia information to a user, at least one of multimedia information including video and audio, still image, text or document which requires continuous reproduction in time. Of the material stored in the material storage means 10, and for the material stored in the material storage means 10 that requires continuous reproduction in time, the time fragment of the material is used as a segment. For materials that do not require continuous playback, add a time interval to continue playback to make pieces, and follow pieces based on synchronization between pieces, continuous temporal connection, or user instruction. The scenario generation means 40 for generating a selection relationship for selecting and reproducing any of the plurality of pieces as a scenario, and the scenario generation means 40 The scenario storage means 20 for storing the created scenario and one of the scenarios stored in the scenario storage means 20 are read and interpreted, and the stored material is extracted based on the relationship between the described pieces. It has a reproducing means 60 for reading and reproducing a desired material.

Further, the scenario generating means 40 described above has a segment or a reproduction effect segment in which temporal and spatial attributes at the time of reproducing the scenario are described, which have a synchronous relationship with the segment or the scenario. It includes a reproduction effect imparting means that is imparted to the scenario as a piece. Further, the above-mentioned reproduction effect giving means includes a changing means for correcting / changing the attribute of the segment based on the situation of the user and the usage situation of the material to give the reproduction effect.

Further, the above-mentioned reproduction effect imparting means has at least one of reproduction time, reproduction speed, and position information as a reproduction effect element regarding the reproduction attribute of the video element or the still pixel element with respect to the video material or the still image material. Including first defining means for defining The reproduction effect imparting means defines at least one of volume, reproduction speed, pitch, sampling frequency, reproduction time, and sound quality information as a reproduction effect element with respect to the reproduction attribute of the speech element with respect to the audio material. 2 definition means are included.

Further, the reproduction effect imparting means, as the reproduction effect element, has a window position, a window size, a reproduction attribute of the text material to the text material,
It includes a third defining means for defining at least one of a font, a column structure and a time interval. Also, the above-mentioned reproduction means 60
Reads the stored material from the material storage means 10,
The scenario is read from the scenario storage unit 20, the reproduction effect described in the reproduction effect piece is interpreted as an appropriate reproduction effect attribute for each material medium, and the reproduction effect element is described in addition to the original reproduction attribute of the material. It includes an interpreting means for reproducing the fragment with the desired effect.

According to the present invention, the reproduction effect imparting means imparts the reproduction effect element to the interpretation according to the material medium. For example, as the reproduction attribute of the video element, the screen position, the reproduction speed, the size, The definition, the number of colors, the reproduction attribute of the voice segment is the volume, the reproduction speed, the pitch, and the sampling frequency, and the reproduction attributes of the still pixel segment are the screen position, the size, the degree of detail, the number of colors, and the text element. As the reproduction attribute of the piece, a window position, a window size, a font, a column structure, and a description of the time interval for reproducing the piece can be defined.

Therefore, according to the multimedia information providing system of the present invention, it is possible to easily realize a multimedia information system capable of switching the effect of the presentation depending on the situation of the user and the scene in which it is used. The flow of presentation and the presentation effect are created as a scenario independently of the application program, and are stored and managed in the scenario storage means.
Based on the playback effect segment described in the scenario, it is possible to select the segment playback effect depending on the scene and the user's situation, and perform effective presentation by changing the playback attribute of each segment. Become.

As described above, in the multimedia information providing method and system of the present invention, any presentation effect can be described as a scenario.

[0021]

FIG. 3 shows the configuration of a multimedia information providing system of the present invention. The system shown in the figure comprises a scenario editing unit 40, a scenario interpretation unit 50, a reproduction unit 60, a data management unit 30, a scenario storage unit 20, and a media data storage unit 10.

The media data storage unit 10 stores multimedia materials such as video, audio and still images. In the scenario storage unit 20, the multimedia editing material stored in the media data storage unit 10 by the scenario editing unit 40 is a material that needs to be reproduced continuously in time.
A time fragment is defined as a segment, and for a material that does not require continuous playback in terms of time, a time interval for continuing playback is added and defined as a segment, and these segments are synchronized or continuous. A scenario in which a temporal connection relation and a selection relation for selecting and reproducing any one of a plurality of subsequent pieces are stored is stored.

The data management unit 30 ensures that the media data and the scenario data stored in the media data storage unit 10 and the scenario storage unit 20 are normally stored even if data addition, update and deletion are repeated. Manage. The scenario editing unit 40 defines the above-mentioned segment for multimedia material and describes it as a scenario that describes the relationship between the segments.

The scenario interpreting unit 50 interprets the synchronization between the pieces, the continuous temporal connection relation and the selection relation described in the scenario, and can reproduce the corresponding media such as video and audio. Get an array of pieces in the time direction. The reproduction unit 60 reads the material from the media data storage unit 10, reproduces the multimedia material such as video and audio based on the scenario interpreted by the scenario interpretation unit 50,
indicate.

In the system having the above configuration, multimedia information is edited and presented to the user by the following procedure. (1) The scenario editing unit 40 is added to the multimedia material stored in the multimedia data storage unit 10 in advance.
In, the definition of the segment and the relationship between the segments are described as a scenario, and the described scenario is stored in the scenario storage unit 20.

Such an editing operation is usually carried out step by step, and data addition, update, and deletion are repeatedly executed, so that the data management unit 30 controls so that the stored data does not become inconsistent. . (2) The expanded array of pieces is passed to the reproducing unit 60, and the reproducing unit 60 reads out a necessary portion of the material stored in the media data storage unit 10 from the information defined in the piece. Reproduce.

[0027]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 4 is a diagram for explaining a scenario of one embodiment of the present invention, and FIG. 5 is a diagram for explaining a video material indicated by a video element appearing in the scenario of one embodiment of the present invention. FIG.

In FIG. 5, the horizontal axis indicates the frame number. Here, assuming that the reproduction speed is fixed at 10 frames per second, the reproduction of this segment starts at the frame number 100. It is played back 10 seconds after being played. This image is an image in which a bird (seagull) is photographed in the frame numbers 0 to 99, and a plant (palm) is photographed in the frame numbers 100 to 199. Similarly, frame numbers from 200 to 299
Until the bird (chicken), frame number 300 to 4
Up to 00, plants (palms) are photographed.

In the scenario shown in FIG. 4, the video material shown in the explanatory view of FIG. 5 is defined as a video segment to which an attribute of a reproduction speed of 30 frames / second is added. FIG.
A solid line with an arrow in the middle indicates a synchronization relationship, a broken line with an arrow represents a selection relationship, and a small circle mark is an auxiliary symbol for making the notation easy to see. Therefore, in this embodiment,
5 is selected by using either the reproduction effect element b described as bird viewing or the reproduction effect element c described as plant application and using the selected reproduction effect element a. It is described to play.

Here, the reproduction effect element a used.
All specify the video reproduction speed, and FIG. 6 shows the specified contents of the reproduction effect element b described as bird viewing.
FIG. 7 shows the specified contents of the reproduction effect element c described as being for plant appreciation. The prescribed content shown in FIG. 6 is that the reproduction speed is 30 frames / sec for the frame numbers 0 to 99, the reproduction speed is 60 frames / sec for the frame numbers 100 to 199, and the reproduction speed is 200 frames to 299 for the frame numbers. Playback speed is 30 frames / sec, frame number is 300-40
Up to 0, the reproduction speed is 60 frames / second.

Similarly, in FIG. 7 as well, the reproduction speed is defined by separating the frame numbers. Next, a flow of interpreting the scenario of FIG. 4 by the scenario interpretation unit 50 shown in FIG. 3 is shown.

In this scenario, the two reproduction effect pieces b and c for bird viewing and plant viewing have a selective relationship. Which reproduction effect element is selected at the time of reproduction is not the gist of the present invention, but the reproduction effect element can be easily changed by a variable held in the system or an explicit or implicit selection instruction from the user. The piece shall be selectable.

Since the selected reproduction effect element has a synchronous relationship with the video element, the attribute of this video element is changed according to the attribute of the selected reproduction effect element. If the reproduction effect segment b for bird viewing is selected, the image segment is reproduced by changing the reproduction attribute of the image segment previously given (reproduction speed = 30 fps) to the reproduction speed shown in FIG. That is,
The video of FIG. 5 has a reproduction speed of 30 frames / second for frame numbers 0 to 99, a reproduction speed of 60 frames / second and a frame number of 20 for frame numbers 100 to 199.
The reproduction speed is 30 frames / second from 0 to 299, and the reproduction speed is 60 frames / second from the frame number 300 to 400.

The image reproduced from the relationship between FIG. 5 and FIG.
In the section where the birds (seagulls and chickens) are being shot, the playback speed is 30 frames / second, which is the same as the shooting speed.
The section where the plant (palm) is photographed is reproduced at high speed at double speed of 60 frames / sec.

As a result, the user is presented with an image as if it were an image taken to explain birds. Similarly, when the image of FIG. 5 is reproduced by using the reproduction effect element for plant appreciation shown in FIG. 7, the section in which the birds are photographed is reproduced at high speed, so that the same image is obtained. The images are presented as if they were taken to explain the plants.

In this embodiment, an example in which the reproduction speed is defined in the reproduction effect element to change the attribute relating to the reproduction speed of the video material by the reproduction effect element has been described, but the present invention is not limited to this example. Alternatively, a method of designating the playback time can be realized. Further, with respect to the reproduction attribute values other than the reproduction time, it is possible to obtain various reproduction effects by modifying / changing the attributes of the pieces with the reproduction effect pieces as in the present embodiment.

It is also possible to specify a function for obtaining the reproduction attribute value, instead of directly specifying the reproduction attribute value. Further, as the reproduction attribute value, the position on the screen when the video segment or the still pixel segment is reproduced, that is, the position information regarding the screen layout, the size, the definition, the number of colors, the volume for the voice segment, and the reproduction speed. , The pitch, the sampling frequency control information, and the like can be defined as a reproduction effect piece, and can be applied to the piece to be reproduced. In addition, window positions, window sizes, fonts, column configurations, time intervals, etc. are defined as the reproduction attributes of the text segment, and the scenario interpretation unit 50 interprets the defined playback effect segment, and the playback unit 60. It is possible to play the material with.

Therefore, the reproducing section 60 shown in FIG. 3 can be realized without considering these reproduction attribute values. An example of replacing the pre-defined attribute information in the segment with the attribute information defined in the reproduction effect segment is shown, but not only the value is simply rewritten, but the pre-defined attribute information in the user It can be easily realized to correct it with the attribute information of the reproduction effect piece according to the situation or the scene to be used.

For example, the reproduction speed defined for a segment is set to 10
% It is possible to define the attribute information of the reproduction effect piece so as to be faster. It should be noted that the present invention is not limited to the above-described embodiment, but can be variously modified and applied within the scope of the claims.

[0040]

As described above, according to the multimedia information providing method and system of the present invention, the presentation effect of the segment to be reproduced can be described in the scenario, so that the presentation effect can be switched depending on the situation of the user. It is possible to easily realize a possible multimedia information providing system.

That is, the effect of the presentation is stored as a scenario in the scenario storage means created independently of the application program and managed,
Even if any presentation effect is described in the scenario, it can be handled by only one application program, and the application program can be created regardless of the content of the presentation.

On the other hand, the content and flow of the presentation,
The scenario editor who creates the presentation effect can create and describe the contents, flow, and presentation effect of the presentation by editing the scenario of the presentation, and change the presentation content. That is, the presentation can be changed without changing the application program.

By managing the reproduction effect as a fragment separately from the material, it is possible to improve the reusability of the editing effect of the multimedia information.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a block diagram of a multimedia information providing system of the present invention.

FIG. 4 is a diagram illustrating a scenario according to an embodiment of the present invention.

FIG. 5 is a diagram for explaining a video material indicated by a video element appearing in a scenario according to an embodiment of the present invention.

FIG. 6 is a diagram (No. 1) for explaining a reproduction effect element that appears in a scenario example of an embodiment of the present invention.

FIG. 7 is a diagram (No. 2) for explaining a reproduction effect piece that appears in a scenario example of an embodiment of the present invention.

[Explanation of symbols]

 10 media data storage unit, material storage unit 20 scenario storage unit, scenario storage unit 30 data management unit 40 scenario editing unit, scenario generation unit 50 scenario interpretation unit 60 playback unit, playback unit

Claims (12)

[Claims] 1. A multimedia information providing method for providing multimedia information to a user, wherein multimedia information including video and audio, a still image, text or a document which requires temporally continuous reproduction is used as a material, A multimedia information providing method, characterized in that an effect corresponding to the reproduction of the material is added to a scenario segment to generate a scenario, and the material is reproduced based on the generated scenario. 2. At least one of said multimedia information
Is stored as a material, and for the material that requires continuous playback in time with respect to the stored material, the time fragment of the material is used as a segment, and continuous playback in time is unnecessary. As for the material, a time interval for continuing the reproduction is given as a piece, and the pieces are synchronized between the pieces, or are connected to each other in a continuous temporal relation, and a plurality of pieces that follow based on a user's instruction. Select and play the selected relationship as a scenario, store it, read one of the stored scenarios, interpret the relationship between the described fragments, and store it. The multimedia information providing method according to claim 1, wherein a desired material is read out from the existing material and reproduced. 3. When describing a scenario to be stored, a segment or a replay effect segment describing temporal and spatial attributes when replaying the scenario is synchronized with the replay effect segment or scenario. The multimedia information providing method according to claim 2, wherein the multimedia information providing method is provided as a fragment having a. 4. When the stored material is read and reproduced, the reproduction effect described in the reproduction effect segment is interpreted as an appropriate reproduction effect attribute for each material medium, and the original reproduction attribute of the material. 4. The multimedia information providing method according to claim 2 or 3, wherein in addition to the above, the segment reproduction in which the desired effect described in the reproduction effect segment is added is performed. 5. Playing back a material that stores a scenario in which any one of a plurality of playback effect pieces can be selected when performing the playback of the piece and presents it according to a user's request or a situation at the time of playback. The multimedia information providing method according to claim 4, wherein the effect is changed. 6. A multimedia information providing system for providing multimedia information to a user, wherein at least one of multimedia information including video and audio, still image, text or document which requires temporally continuous reproduction is provided. For the material storing means for storing as a material and the material stored in the material storing means, for the material that requires continuous reproduction in time, the time fragment of the material is used as a segment and continuous in time. For the material that does not need to be reproduced, a time interval for continuing the reproduction is given as a segment, and a plurality of subsequent segments are synchronized based on synchronization between the segments, continuous temporal connection relationship, or user instruction. Scenario generating means for generating a selection relationship for selecting and reproducing any of the pieces of the segment, and the scenario generating means. A scenario storage unit that stores a scenario and one of the scenarios stored in the scenario storage unit is read and interpreted, and a desired one of the stored materials is selected based on the described relationship between the pieces. A multimedia information providing system, comprising: a reproducing unit for reading and reproducing a material. 7. The scenario generation means uses a segment or a reproduction effect segment in which temporal and spatial attributes at the time of reproducing the scenario are described as a segment having a synchronous relationship with the segment or the scenario. 7. The multimedia information providing system according to claim 6, further comprising a reproduction effect giving means for giving to the scenario. 8. The reproducing effect adding means includes a changing means for correcting / changing the attribute of the element based on the situation of the user and the usage scene of the material to give a reproducing effect. Described multimedia information providing system. 9. The reproduction effect imparting means includes, as the reproduction effect element, at least one of reproduction time, reproduction speed, and position information regarding a reproduction attribute of a video element or a still pixel element with respect to a video material or a still image material. 8. The multimedia information providing system according to claim 7, further comprising a first defining means for defining. 10. The reproduction effect imparting means sets, as the reproduction effect element, at least one of volume, reproduction speed, pitch, sampling frequency, reproduction time, and sound quality information regarding a reproduction attribute of the speech element with respect to an audio material. 8. The multimedia information providing system according to claim 7, further comprising second defining means for defining. 11. The reproduction effect imparting means, as the reproduction effect element, has at least one of a reproduction position of a text material with respect to a text material, a window position, a window size, a font, a column structure, and a time interval.
8. The multimedia information providing system according to claim 7, further comprising third defining means for defining one. 12. The reproducing means reads the material stored in the material storing means, reads a scenario from the scenario storing means, and reproduces the reproducing effect described in the reproducing effect segment for each material medium. 7. The multimedia information provision according to claim 6, further comprising an interpreting unit that interprets as an appropriate reproduction effect attribute, and reproduces a segment added with a desired effect described in the reproduction effect segment in addition to the original reproduction attribute of the material. system.