JP3409374B2 - Data output device and method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention inputs a plurality of media data such as video signals and audio signals from a plurality of reproducing devices such as a video disc player and a video tape recorder, and stores them at a predetermined position on a common display. A data output device and method used when displaying a video signal at a predetermined time, in particular,
The present invention relates to a data output device and a method thereof, by which a user can efficiently perform a temporal layout design when reproducing reference media data.

[0002]

2. Description of the Related Art A multimedia system is known in which, for example, video signals from a plurality of media are displayed at a predetermined position on a common display screen at a predetermined timing. Such a multimedia system usually has a function that allows the user to interact with the system. A typical example of the multimedia system is that the user is provided with data related to a part of the content of the data being reproduced. Has the ability to see and play.

FIG. 10 is a diagram for explaining a case where a user displays reference media data related to a part of contents of media data being displayed on a display screen of a multimedia system. As shown in FIG. 10, there is a reference button 102 in a partial area of the media data 101 displayed on the display, and when the user specifies the reference button 102 using a mouse or the like, the reference button 102 corresponding to this reference button 102 is displayed. The media data 103b is displayed. Before the reference data 103b and the reference button 102 are designated, they are not displayed on the display (reference media data 103a).

In the conventional multimedia system, the layout on the display when reproducing the media data and the reference media data is determined (designed) as follows. For example, the multimedia system is shown in FIG.
A table 201 as shown in (A) is created, and the time at which the media data from each media is displayed (reproduced) is allocated on the reproduction time axis 202 common to each media data to determine the temporal layout. . FIG. 11 (A)
Then, for example, the playback time 204 of "Media 2" is
"T3" to "t8".

Also, the multimedia system determines the layout of the display area of each medium on the display screen, as shown in FIG. 11B, for example.

[0006]

However, in the multimedia system described above, the reference media data 103a as shown in FIG. 10 appears on the display screen when the corresponding reference button 102 is clicked by the user. , There is a problem that the display time cannot be pre-allocated on the reproduction time axis 202 like “reference medium 3” shown in FIG. In addition, in the above-described multimedia system, from the layout of the display area of each medium on the display screen shown in FIG. There is also a problem that it is difficult to distinguish from the reference media data whose existence and reference time are unknown.

The present invention solves the above-mentioned problems of the prior art, and even for reference media data whose reference time is unknown,
An object of the present invention is to provide a data output device and a method thereof that can appropriately perform temporal layout design. Further, the data display method of the present invention provides a data output device and method capable of easily identifying reference media data and media data of initial reproduction in layout design for determining a display area on a display screen. The purpose is to

[0008]

In order to solve the problems and achieve the above-mentioned objects in the above-mentioned conventional technique, the data output device of the present invention further comprises a converting means for converting a plurality of media data into a predetermined format. An input unit for inputting a reference instruction and an output mode instruction indicating an output mode of the converted media data; and corresponding to the reference command of the media data from the conversion unit in response to the reference command from the input unit. Display time of the reference media data, based on the timing of inputting the reference instruction, relative display time defined using a time axis uniquely determined for the reference media data included in the output mode instruction And a control unit that creates an output instruction indicating that the output is to be output based on the converted display time, and an output finger from the control unit. Depending on, and an output unit for performing an output related to the display and the display based on the display time.

Further, the control means of the data output apparatus of the present invention, for example, when the reference instruction is inputted, the reference media data is outputted by the output means when the reference instruction is inputted. If the original media data is output at the same time or the reference instruction is input, the output of the reference source media data is temporarily stopped, the reference media data is output, and after the reference media data output is completed. The output of the referencing media data is restarted.

[0010]

In the data output device of the present invention, the user inputs the output mode instruction from the input means. Then, the user inputs the reference instruction from the input unit, for example, in a state where the output unit outputs the reference-source media data. When the reference instruction is input, the control means defines the reference media data corresponding to the reference instruction by using a time axis uniquely determined for the reference media data included in the output mode instruction. The relative display time is converted into a display time based on the timing when the reference instruction is input, and an output instruction indicating that the relative display time is output is created based on the converted display time. Then, the output means performs a display based on the display time and an output related to the display according to the output instruction from the control means.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The data output device of the present invention will be described below with reference to specific examples. FIG. 1 is a block diagram of a multimedia system as a data output device of the present invention. As shown in FIG. 1, the multimedia system includes n information sources 1-1.
.. 1-n, a conversion unit 2, a control unit 3, a drive unit 4, an input unit 6 and an output unit 5. The information sources 1-1 to 1-n output media data such as video signals or audio signals to the conversion unit 2. The conversion unit 2 has information sources 1-1 to 1
The media data from -n is converted into a predetermined format and output to the control unit 3. Based on the instruction signal from the input unit 6, the control unit 3 creates the display signal S3a from the media data input from the conversion unit 2 and outputs the display signal S3a to the output unit 5. The output unit 5 includes a monitor 5a for displaying a video signal included in the media data and a speaker 5b for outputting an audio signal, and displays and outputs according to the display signal S3a from the control unit 3. To do. The input unit 6 includes a keyboard, a mouse, a pen, and the like, and outputs to the control unit 3 an instruction signal S6 according to a reference instruction and an output mode instruction from a user having a description format as described below.
The drive unit 4 outputs a control signal to the information sources 1-1 to 1-n based on the control signal S3b from the control unit 3,
These are driven at a predetermined timing.

In this multimedia system, in order to display the media data output from the information sources 1-1 to 1-n on the output unit 5 at a predetermined position at a predetermined timing, the user inputs to the input unit 6. As will be described later, an instruction having a description format is input, and an instruction signal S6 corresponding to this instruction is input.
Is output from the input unit 6 to the control unit 3. When the control unit 3 receives the instruction signal S6, the control unit 3 drives the predetermined information sources 1-1 to 1-n at a predetermined timing via the drive unit 4 according to the content of the instruction signal. The information sources 1-1 to 1-n driven by this output either the video signal or the audio signal of the media data to the conversion unit 2.
The conversion unit 2 converts a video signal or an audio signal input from the information sources 1-1 to 1-n into a predetermined format,
Output to the control unit 3. When the video signal is input via the conversion unit 2, the control unit 3 inputs this video signal into the input unit 6
The signal is output to the output unit 5 at a predetermined timing based on the instruction signal S6 from the display unit 5 and is displayed at a predetermined position on the monitor 5a. Further, when the voice signal is input through the conversion unit 2, the control unit 3 outputs the voice signal to the output unit 5 at a predetermined timing based on the instruction signal S6 from the input unit 6,
Output from the speaker 5b.

The user operates the input unit 6 in order to display or output the media data from a desired information source among the information sources 1-1 to 1-n at a desired display position at a desired time. Design the layout.

Next, in the multimedia system of this embodiment, a procedure when a user designs a layout and a process in the multimedia system will be described. The procedure in this case is that the user inputs the information sources 1-1 to 1- 1 shown in FIG. 1 to the control unit 3 via the input unit 6.
Outputting the media data from n on the monitor 5a of the output unit 5 and instructing the display timing and display position. In the multimedia system of this embodiment,
In order to allow the user to efficiently perform the procedure involved in layout design, the concept of "relative display time axis" is introduced as a time axis that is uniquely determined for reference media data. In the multimedia system of the present embodiment, when the user clicks (specifies) a reference button assigned to a part of the media data on the display screen with a mouse or the like, the reference media data corresponding to the reference button is displayed on the screen. Displayed above. The “relative display time axis” is the control unit 3
Is a display time axis used when designating the timing when this reference media data is output to the output unit 5. For each reference layout to which each reference media data belongs, the reference button of the layout that is the reference source is It is set based on the clicked time. The time defined based on this relative display time axis is called “relative display time”.

FIG. 2 is a diagram for explaining the concept of the "relative display time axis". In FIG. 2, the display time axis 20 is a display time axis relating to “media data A”, and is assigned to the absolute display time axis used when displaying the media data on the monitor 5a. The relative display time axis 22 is a display time axis regarding “reference media data B” with “media data A” as a reference source, and time on the display time axis 20 indicating the timing of clicking the reference button in “media data A”. The starting point is "ta". The relative display time axis 26 is related to the “reference media data D”, and similarly to the case of the “reference media data B”, with the “media data A” as the reference source, the time “tb” on the display time axis 20 is the starting point. There is. The relative display time axis 24 is “reference media data B”
Is a display time axis related to the “reference media data C” whose reference source is, and the time “t1a” on the relative display time axis 22 indicating the timing of clicking the corresponding reference button in the “reference media data B” is the starting point.

The time on the display time axis (including the relative display time axis) regarding the reference source media data (including the reference media data), which is the starting point of the relative display time axis, is between the start time and the end time. Exists in. When the start time of the display time axis related to the reference source media data is Ts and the end time is Te, the playback time zone indicating the range of time when the reference button is pressed on the display time axis related to the reference source media data is as follows. It is defined as the description (1-1). Reproduction time zone of reference button ≡ (Ts, Te) (1-1) At this time, if the time when the reference button is pressed is T on the display time axis regarding the reference source media data, the following relationship is satisfied. . Ts ≦ T ≦ Te (2-1) In the above relationship (2-1), T is an indefinite value.

On the other hand, the relative display time axis relating to the referenced media data is relTs for the start time and re for the end time.
In the case of 1Te, the reproduction time zone on the relative display time axis is defined as described in (1-2) below. Layout time zone of media data on relative display time axis ≡ (relTs, relTe) (1-2) When the time zone shown in the above description (1-2) is converted to the display time axis relating to the reference media data It becomes like the following description (1-3). Reproduction time zone at the reference source of media data ≡ (relTs + T, relTe + T) (1-3) where T in the above description (1-3) is the above description (2
-1) is satisfied. Normally, the timing at which the reference button is clicked by the user is unknown, so by defining the time at which the reference media data is displayed based on the relative display time axis starting from the timing at which the reference media data is clicked. The user can independently determine the temporal layout regarding the reference media data separately from the temporal layout of the reference source media data.

Next, the syntax (descriptive grammar) of the output mode instruction input by the user to the input unit 6 will be described.
This instruction is described mainly using the following scenes, media data, referenced scenes, layouts, and referenced layouts. (1) Scene A scene is a target for the control unit 3 to display the media data from the conversion unit 2 on the monitor 5a of the output unit 5, and to display the media data on the monitor 5a. It is a description that defines the two-dimensional area to be obtained. At this time, the media data is sent to the monitor 5 of the output unit 5.
The time displayed on a is defined based on the absolute time axis defined by the output unit 5, and the two-dimensional area that can be the target for displaying the media data is defined by the coordinates in the xy coordinate system defined on the monitor 5a. To be done. FIG. 3A is a diagram for explaining the concept of a scene. For example, as shown in FIG. 3A, scene 1 is a scene cut (sc1) indicating time “t1” on the display time axis 20.
And a scene cut (sc2) indicating the time “t8”, and x− that can be a target for displaying the media data.
Coordinates (100, 10) indicating a two-dimensional area in the y coordinate system
0) is defined. The scene is defined by the predicate notation as shown in the following description (3-1).

[0019]   scene (<scene identification name>, <display area>,   [<Scene cut 1>, ..., <Scene cut N>]) (3-1)

As shown in the description (3-1), <scene>
Is a description including <scene identification name>, <display area>, and a set of a plurality of scene-cuts.

Here, the scene cut indicates the time when the media data is displayed (reproduced) on the monitor 5a, and is defined as the following description (4).

[0022]   <Scene cut> ≡ <Play time> (4)

<Scene cut 1> described first in <Scene cut> is first output to the output unit 5 by the control unit 3 among the media data from the information sources 1-1 to 1-n. The output start time of the media data
<Scene cut n> indicates the output end time of the media data that is finally output. Also, the number of <scene cuts> is arbitrary. If the above description (3-1) is used, FIG.
Scene 1 shown in (A) is described as in the following description (3-2). scene (scene 1, 100, 100, [sc1, sc2]) (3-2) At this time, sc1 = “t1”, sc2 = “t8”, and “100, 100” indicates a <display area>.

(2) The reproduction time at which the media data information sources 1-1 to 1-n output (reproduce) the media data to the control unit 3 via the conversion unit 2 is the information source 1
-1 to 1-n are defined using a reproduction time axis uniquely defined. Media data is defined by the predicate notation as shown in the following description (5). media (<media identification name>[<media cut 1>, ..., <media cut M>]) (5) As shown in the description (5), the media data includes the information sources 1-1 to 1-1 shown in FIG. 1-n, that is, a description including <media identification name> for specifying a medium and a set of a plurality of media cuts.

The media cut defines the reproduction time on the reproduction time axis of the corresponding medium. <Media cut> ≡ <Play time> (6)

On the reproduction time axis of the medium, the media cut 1 indicates the reproduction start time and the media cut M indicates the reproduction end time. The number of media cuts is arbitrary.

The scene cut indicates the time on the absolute display time axis in the output unit 5, while the media cut indicates the time on the reproduction time axis defined in the corresponding medium.

Reference media data and a reference button are also treated as media data. (3) Referenced scene (referenced-
The reference scene is a reference scene cut indicating the time when the control unit 3 causes the reference media data from the conversion unit 2 to be displayed on the monitor 5a of the output unit 5, and a target for displaying the reference media data on the monitor 5a. It is a description that defines the two-dimensional area to be obtained. At this time, the time at which the reference media data is displayed is the “relative display time” defined based on the relative display time axis independently defined for the reference scene, and is a two-dimensional object that can be the target for displaying the reference media data. Area is monitor 5
a defined by coordinates in the same xy coordinate system as the scene defined above. The relative display time axis is the concept described with reference to the relative display time axes 22, 24, and 26 in FIG. 2, and corresponds on the display time axis or the relative display time axis defined in the reference source scene or the referenced scene. The display time axis is defined based on the time when the reference button is clicked. FIG. 3B is a diagram for explaining the concept of the reference scene. For example, as shown in FIG. 3B, the reference scene 1 indicates the relative display time "t0a" for displaying the reference media data on the monitor 5a, and the reference scene cut (ref_sc1) indicates the relative display time "t5a". The reference scene cut (ref_sc2), the reference scene cut (ref_sc3) indicating the relative display time “t8a”, and the coordinates (100, 100) indicating the two-dimensional area in the xy coordinate system that can be the target of displaying the reference media data. 100). At this time, the relative display time "t0a", "t5a", "t8a"
Is the time on the relative display time axis 22 defined with reference to the time “t2” on the display time axis 20 of scene 1.

The reference scene is described below (7-
It is defined by the predicate notation as shown in 1).

[0030]   referenced_scene (       <Reference scene identification name>,       <Indicated Area>,       [<Reference scene cut 1>,               ..., <Reference scene cut N>]) (7-1)

As shown in the description (7-1), the reference scene includes a <reference scene identification name>, a <display area>, and a plurality of reference scene cuts (r
It is a description including "effected-scene-cut". That is, in the reference scene, in the display space provided by the scene to which the reference source media data belongs,
The attributes of the display space provided when the reference media data is referred to are described.

For example, when making a reference as shown in FIG. 2, the user inputs the media data A to the input unit 6.
An instruction describing a scene A defining a display space for displaying and reference scenes B, C, D defining a display space for displaying each of the reference media data B, C, D is input.

If the above description (7-1) is used, FIG.
The reference scene 2 shown in is described as in the following description (7-2). referred_scene (ref_scene1, 100, 100, [ref_sc1, ref_sc2, ref_sc3]) (7-2) At this time, “ref_scene1” is the identification name of the reference scene 1. Also, ref_sc1 = “t
0a ”, ref_sc2 =“ t5a ”, ref_sc3
= “T8a”. Further, “100, 100” indicates a <display area>.

(4) Layout The layout is a description that assigns each media cut defined by the above-mentioned media data to each scene cut defined by the above-mentioned scene. When the number of scene cuts is N, the layout is defined by the predicate notation as shown in the following description (8-1). layout ([Layout on scene cut 1], ... [Layout on scene cut N]) (8-1) The layout is a set of layouts on each scene cut as shown in the above description (8-1). Described in. Here (1 ≦ i ≦ N), where the number of media cuts assigned to the scene cut i is M, [layout on scene cut i] is defined as in the following description (9). [Layout on scene cut i] ≡ [<scene cut i>, [modification of media cut 1 assigned to scene cut i], ... [Modification of media cut M assigned to scene cut i]] (9) As shown in the above description (9), a plurality of media cut modifiers can be assigned to each scene cut.

The modification of the media cut j is described below (1
0), it is defined by <media cut j> and <attribute value change description>. [Modification of media cut j] ≡ [<media cut j>, [attribute value change description]] (10) The [attribute value change description] includes a placement area (position and size) of the media cut j on the display. Indicates a change in the attribute value of the.

(5) Reference layout (ref
erenced_layout) Referenced layout is a description that defines that each media cut defined by the above-described media data is assigned to the above-mentioned referenced scene. When the number of reference cuts is N, the reference layout is defined by the predicate notation as shown in the following description (11).

[0037]   referenced_layout (<reference source scene>, <button>, <reference> Illumination mode>, [[Layout on reference scene cut 1], ...             , [Layout on Referenced Scene Cut N]])                                                               (11)

The reference layout is as described above (1
As shown in 1), <reference scene>, <button>,
It is described by a set of <reference mode> and [layout on reference scene cut]. Here, when the number of media cuts assigned to the reference scene cut i (1 ≦ i ≦ N) is M, [layout on the reference scene cut i] is described below (12).
Is defined as

[0039]   [Layout on reference scene cut i] ≡ [<Referenced scene cut i>, [referenced scene cut i Modification of attached media cut 1], ... [referenced scene cut i Modification of media cut M assigned to]] (12)

As described in the above description (12), a plurality of media cuts can be assigned to each reference scene cut, as in the layout described above. <Reference source scene> is a description of a reference source scene in which the reference media data laid out by the reference layout is referred to. <Button> is a description for specifying a reference button in the reference source scene. The <reference mode> is a description showing the reproduction mode of the reference media data, and has a “simultaneous reproduction mode” and an “interruption reproduction mode”.

In the "simultaneous reproduction mode", the reference media data is reproduced in parallel with the reproduction of the reference source media data by pressing (clicking) the reference button. In this case, the reproduction area of the reference media data and the media data of the reference source may conflict with each other. In that case, the user is willing to actively refer to the reference media data. Therefore, basically, the reference media data is preferentially reproduced.

The "interrupt reproduction mode" is a mode in which the reproduction of the reference source media data is stopped and the reference media data is reproduced. When the reproduction of the reference media data is completed, the reproduction of the reference source media data is resumed. To do. During the reproduction period of the reference media data, the reproduction of the reference source media data is stopped. As this "interrupt playback mode", in addition to the above-described mode, a mode in which the playback of the reference source media data is excluded when the reference media data is played back can be considered, but the reference action is only reading the reference media data. Without
Considering that there is a match with the reference source media data, basically, the reference media data may be preferentially reproduced, and the stopped reference source media data may be reproduced.

The description of the instruction expressing the layout image as shown in FIG. 4 will be described below. The layout image shown in FIG. 4 is an image using the scene 1 shown in FIG. 3A and the reference scene 1 shown in FIG. 3B. FIG. 4 shows the instructions representing the layout image in the following descriptions (A1) to (A8).

[0044]

[Table 1]

(A1) This is a description related to scene 1, and corresponds to the above description (3-2). (A2) This is a description regarding Referenced 1 and corresponds to the above description (7-2).

(A3) "med" as media data
ia1 ”is a description based on the above description (5), and media cuts are“ mc11 ”and“ mc1 ”.
Have 2 ". (A4) This is a description based on the above description (5) regarding "button1" as media data, and has "bc11" and "bc12" as media cuts. (A5) “ref_media” as media data
1) is a description based on the above description (5) regarding “1”,
As media cut, "rmc11", "rmc1"
Have 2 ". (A6) “ref_media” as media data
2) which is based on the above description (5),
For media cut, "rmc21", "rmc2"
Have 2 ".

(A7) As shown in FIG. 4, the media cuts of "media1" and "button1" described in (A3) and (A4) are assigned to each scene cut of scene 1. Here, "sc
"Mc11" and "bc11" are assigned to "1", and "mc12" and "b" are assigned to "sc2".
The modification of "c12" is assigned. "Mc11", "mc1"
The modifications “2”, “bc11”, and “bc12” are modifications relating to the display area.

(A8) As shown in FIG. 4, for each reference scene cut of reference scene 1,
“Ref_medi” described in (A5) and (A6)
It is a description that allocates media cuts of “a1” and “ref_media2”, and is “ref_mc11” for “ref_sc1”, “ref_mc12” and “ref_mc21” for “ref_sc2”, and “ref_mc2” for “ref_sc3”.
2 ”is allocated. Further, in the above-mentioned description (11), “scene1” is described as <reference source scene>, and “button1” is described as <button>.

Here, the reference cut "r"
ef_sc1 ”,“ ref_sc2 ”and“ ref_
“Sc3” is the time on the relative display time axis 22 uniquely determined for the reference scene 1, and the relative display time axis 22 starts from the time “t2” on the display time axis 20 of the scene 1.

The above description (A1)-(A
When an instruction including 8) is input to the input unit 6, the control unit 3 uses the instruction included in the instruction signal S6 from the input unit 6,
Perform predetermined processing. A procedure of processing in the multimedia system of this embodiment will be described. Figure 5
3 is a flowchart showing a procedure of processing in the multimedia system of the present embodiment. Step S11: When the user inputs an output mode instruction described according to the above-described description format from the input unit 6, an instruction signal S6 corresponding to the output mode instruction is output to the control unit 3. Step S12: The media data from the information sources 1-1 to 1-n is output to the conversion unit 2, converted into a predetermined format by the conversion unit 2 and output to the control unit 3. Step S13: The control unit 3 performs the process described below, and the output unit 5 performs display and audio output according to the process.

The processing performed by the control unit 3 in step S13 in FIG. 5 described above will be described. FIG. 6 is a flowchart of the processing performed by the control unit 3. Step S1: The control unit 3 uses the {initial layout} as the "reproduction layout group". {Initial layout}
Is the layout described in the instruction included in the instruction signal S6, and in the case of FIG. 4, “lay” in the above description (A7).
"out" is the {initial layout}.

Step S2: The control section 3 judges whether or not the "reproduction layout group" is an empty set {}, and if so, the processing is ended (end), and if not, the step is ended. The process of S3 is executed.

Step S3: The control section 3 performs reproduction according to the layout included in the "reproduction layout group", and causes the output section 5 to display an image according to the layout. In addition,
When performing playback according to the reference layout, the relative display time specified with reference to the relative display time axis used for the reference scene is used for the <reference scene> described in the reference layout. Based on the time when the reference instruction on the display time axis is input, it is converted into the display time on the absolute display time axis determined by the output unit 5. Then, the reference media data is displayed on the monitor 5a based on the converted display time.

Step S4: The control section 3 judges whether or not the reference button has been pressed. If it is judged that the reference button has been pressed, the processing of step S5 is executed, and if it is judged that the reference button has not been pressed, the step is carried out. The process of S6 is executed. For example, in the case of FIG. 4, it is determined whether or not the reference button is clicked on the reproduction screen provided by the scene 1.

Step S5: The processing in step S5 is shown in FIG. Step S5-1: The control unit 3 determines whether the <reference mode> of the reference layout corresponding to the reference button pressed in step S4 is the “interruption reproduction mode” or the “simultaneous reproduction mode”. If it is determined that they are interrupts, step S5-2 is executed, and if they are determined to be simultaneous, step S5-5 is executed. For example, in the case of FIG. 4, “RefMode” in the above description (A8).
Is the "interruption reproduction mode", step S5-2 is executed, and if it is the "simultaneous reproduction mode", step S5 is executed.
-5 is executed.

Step S5-2: The control section 3 stops the reproduction of the layout of the reference source. For example, in the case of FIG. 4, the reproduction of the layout related to scene 1 is stopped. Step S5-3: The control unit 3 "playback layout group"
Add {reference layout} to. For example,
In the case of FIG. 4, the control unit 3 adds the reference layout of the above description (A8) to the “reproduction layout group”. Step S5-4: The control section 3 adds {reference source layout} to the "playback stop layout group". As a result, the layout of the reference source is not displayed.

Step S5-5: The control section 3 adds {referenced layout} to the "reproduction layout group".

Step S6: The control section 3 judges whether or not there is a layout for ending reproduction, and if there is, a process of step S7 is executed, and if not, the process of step S1 is executed again. Step S7: The processing in step S7 is shown in FIG. Step S7-1: The control unit 3 "playback layout group"
Remove {playback end layout} from and add {reference source layout}. Step S7-2: The control unit 3 removes the {reference source layout} from the "playback stop layout group". After the display according to the reference layout is completed by steps S7-1 and S7-2, the display according to the layout serving as the reference source is performed again.

For example, in the case of FIG. 4, when the reference button is clicked at time “t2” on the display time axis 20, when “RefMode” is “simultaneous reproduction mode” in step S5-1 of FIG. 9A, the time “t0” to “t” on the display time axis 20 in the output unit 5 is displayed.
2 ”, an image corresponding to the layout of scene 1 is displayed. Also, at times “t2” to “t8”, images according to the layouts of both scene 1 and reference scene 1 are played back at the same time, but for overlapping portions, images according to the layout of reference scene 1 are given priority. Is displayed. Further, at times “t8” to “t10”, an image according to the layout of the reference scene 1 is displayed.

On the other hand, in step S5-1 of FIG.
When “fMode” is “interruption reproduction mode”, FIG.
As shown in (B), the display time axis 20 in the output unit 5
At times “t0” to “t2”, an image according to the layout of scene 1 is displayed. In addition, time “t2” to “t
In “10”, an image corresponding to the layout of the reference scene 1 is displayed. In addition, time “t10” to “t1
In “6”, among the images according to the layout of scene 1, the remaining images that are not displayed at times “t0” to “t2” are displayed. In the above-described processing, the case where a video signal such as a video signal is displayed as the media data has been described, but the audio signal as the media data related to the video signal is output by the output unit in synchronization with the display of the corresponding video signal. 5 is output from the speaker 5b.

As described above, according to the data output apparatus and the method thereof of this embodiment, in the instruction input by the user to the input unit 6, "simultaneous reproduction mode" and "interruption reproduction mode" are selected as "RefMode". As described above, the desired mode can be selected, and the layout design can be performed based on the display mode according to the preference. Further, according to the data output apparatus and the method thereof of the present embodiment, the display of the reference source image which is an image corresponding to the layout of the scene 1 and the display of the reference image which is an image corresponding to the referenced scene 1 are selectively performed. In addition, it can be performed simultaneously or separately, and a convenient design environment can be provided to the user. Further, according to the data output apparatus and the method thereof of this embodiment, the reference media data is preferentially displayed in the display in the “simultaneous reproduction mode”, so that the user can distinguish the reference media data from the reference source media data. Can be facilitated. Also,
If the "interrupt playback mode" is displayed, the reference media data and the reference source media data are displayed separately.
These can be easily identified.

12 to 15 show examples of the case where the definition of the scene, media data and layout in the above-described embodiment is described by using the logical expression language "Prolog" in the instruction given by the user. FIG. 12 shows the case where the definition of the scene is described using “Prolog”, and it is described that the list processing is performed on “SCs” indicating the list of scene cuts. FIG. 13 shows a case where the definition of media data is described using “Prolog”, and it is described that list processing is performed on “MCs” indicating a list of media cuts. 14 and 15 show the case where the layout definition is described using "Prolog", and "S" showing the list of layouts is shown.
It is described that the list processing is performed on “CLs”.

The instruction input by the user in this embodiment can be described using another logical expression language in addition to the above-mentioned logical expression language "Prolog".

According to the data output apparatus and the method thereof of the present invention, the relative display time of the reference image is defined by using the relative display time axis uniquely defined for the reference screen. Even if the instruction input time is unknown, the relative display time of the reference image can be determined, and the temporal layout of the reference image can be performed independently of the reference instruction input time. Further, according to the data output device and the method thereof of the present invention, by displaying the reference source image and the reference image at the same time, the layout of the reference source image and the layout of the reference image can be associated with each other. . Further, according to the data output device and the method of the present invention, when the reference instruction is input, the display of the reference source image is temporarily stopped to display the reference screen, and after the display of the reference image ends, the reference source is displayed. By displaying the rest of the image, it is possible to easily identify the reference source image and the reference image for layout design. Further, according to the data output device and the method thereof of the present invention, by using the reference image as the reference source image, it is possible to obtain the above-described effect even for the reference image referred to by the reference image.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a multimedia system as an embodiment of a data output device of the present invention.

FIG. 2 is a diagram for explaining the concept of a relative time axis.

FIG. 3A is a diagram for explaining the concept of a scene, and FIG. 3B is a diagram for explaining the concept of a referenced scene.

FIG. 4 is a diagram for explaining a layout image.

FIG. 5 is a flowchart of a processing procedure in the multimedia system according to the present embodiment.

FIG. 6 is a flowchart of a process performed by a control unit in step S13 of FIG.

7 is a flowchart of the process of step S5 of FIG.

FIG. 8 is a flowchart of the process of step S7 of FIG.

9A is a diagram for explaining a display image in the simultaneous reproduction mode, and FIG. 9B is a diagram for explaining an interrupt reproduction mode.

FIG. 10 is a diagram for explaining a display screen in a conventional multimedia system.

11A is a diagram for explaining a temporal layout of media data in a conventional multimedia system, and FIG. 11B is a diagram for explaining a display layout of media data.

FIG. 12 is an example in which the definition of a scene is described using “Prolog”.

FIG. 13 is an example in which the definition of media data is described using “Prolog”.

FIG. 14 is an example in which a layout definition is described using “Prolog”.

FIG. 15 is an example in which a layout definition is described using “Prolog”.

[Explanation of symbols]

1-1 to 1-n ... Information source 2 ... Converter 3 ... Control unit 4 ... Drive unit 5: Output section 5a ... Monitor 5b ... Speaker 6 ... Input section

Continuation of the front page (56) References JP-A-5-81406 (JP, A) JP-A-3-132695 (JP, A) Akihiko Handa Two others, "Multimedia / Hypermedia Coding Standard (MHE G)" Trends ”, IEICE Technical Research Report (DE92-39 to 42 Data Engineering), The Institute of Electronics, Information and Communication Engineers, March 19, 1993, Vol. 92, No. 527, p. 25-31 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 11/80 H04N 5/76 H04N 5/91 H04N 5/262 CSDB (Japan Patent Office)

Claims (5)

(57) [Claims] 1. A converting unit for converting a plurality of media data into a predetermined format, an input unit for inputting a reference instruction and an output mode instruction indicating an output mode of the converted media data, and an input unit for inputting the output mode instruction. In response to the reference instruction, the reference media data corresponding to the reference instruction among the media data from the conversion means is used by using a time axis uniquely determined for the reference media data included in the output mode instruction. The relative display time defined by the reference instruction is converted into the display time based on the input timing of the reference instruction, and the output means is generated based on the converted display time. Output means for outputting the related media data output based on the display time according to the output instruction from Data output device for the butterflies. 2. The control means, when the reference instruction is input, outputs the reference media data at the same time as the reference source media data output by the output means when the reference instruction is input. The data output device according to claim 1, wherein: 3. The control means preferentially displays the reference media data when the reference media data and the reference source media data are displayed in an overlapping manner. The data output device according to claim 2. 4. The control means, when a reference instruction is input,
Claiming that the output of the reference source media data is temporarily stopped, then the reference media data is output, and the output of the reference source media data is resumed after the output of the reference media data is completed. Item 1. The data output device according to item 1. 5. A data output method for outputting a plurality of media data based on display time, comprising inputting an output mode instruction and a reference instruction, and converting into a predetermined format in response to the input of the reference instruction. Of the media data, the reference instruction corresponding to the reference instruction corresponding to the reference instruction is a relative display time defined by using a time axis uniquely determined for the reference media data included in the output mode instruction. Is output based on the input timing, and related media data is output based on the converted display time.