JP2010206803A - Video synthesizing device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video synthesizing device or the like capable of attaining a playback style reflecting the intention of a distributor for the playback style during special playback of picture-in-picture videos. <P>SOLUTION: The video synthesizing device receives input of two or more pieces of video data and management data relating to at least one of the two or more pieces of video data and outputs the playback video of a picture-in-picture form in which the videos of the two or more pieces of video data are synthesized. The management data includes instruction information instructing the playback style during the special playback of the playback video, and a read means for reading the instruction information from the management data and a synthesis means for synthesizing the videos of the two or more pieces of video data in the instructed playback style according to the instruction information read by the read means are provided. The instruction information is the information indicating whether or not the video of a sub-screen is to be displayed during the special playback of the video of a main screen in the playback video of the picture-in-picture form. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video synthesizing apparatus that outputs a plurality of video data and a reproduced video in a picture-in-picture format obtained by synthesizing videos of the plurality of video data.

  The absolute amount of video data that can be handled at once by the user and the video equipment handled by the user has increased dramatically due to the speeding up of the network infrastructure and the increase in the capacity of recording media. Along with this, various functions realized using a plurality of video data and advanced applications based on them are also appearing. One such function is a function called picture-in-picture. “Picture-in-picture” is a function for displaying two images simultaneously by overlapping a small child screen on a screen (parent screen). For example, “multi-angle display” that displays images taken from a different angle from the main screen image in the sub-screen, and “commentary display” that displays additional information related to the main screen image in commentary format (for example, movie It is used to display a director's commentary video, etc., which includes a secret story about the shooting of the film.

  This picture-in-picture function is realized, for example, by a process of decoding two video data on two different layers and superimposing the decoded videos as shown in FIG.

  FIG. 18 shows a difference in the display form of the sub-screen video in the picture-in-picture function. In addition to the rectangular video 1801, a video 1802 having an arbitrary shape may be used for the sub-screen. The picture-in-picture function and implementation method are described in, for example, Patent Document 1 and the like.

JP 2005-123775 A

  In picture-in-picture, two images are reproduced and displayed simultaneously. For this reason, when special playback such as high-speed playback is instructed, for example, when the picture of the parent screen is displayed during picture-in-picture display, whether the child screen video follows the parent screen video or special playback, Or, there is no restriction on whether or not to perform special reproduction, and it depends on the implementation of the reproduction equipment. Therefore, it is not possible to limit the operation during special playback on the side providing the video. Here, special playback refers to a video playback mode different from normal playback, such as slow playback, frame-by-frame playback, and reverse playback, as well as high-speed video playback such as 2 × speed and 4 × speed.

  FIG. 19 shows three typical routes (broadcasting, communication (network), and package distribution) when video content is distributed from a provider to a consumer. Also, as a composite distribution means, the basic part is sent by broadcasting or package distribution, and the additional part is sent by communication (network). When video including picture-in-picture is distributed from the provider to the consumer in this way, depending on the application (application), the video provider may change the playback mode at the time of special playback of picture-in-picture. There is a request that you want to specify as the provider intended.

  For example, the video displayed on the child screen is a privilege video, and there is a request to set as a viewing condition that the video is viewed at the end without being played back at high speed. On the other hand, in the conventional picture-in-picture, the playback mode at the time of special playback is left to the playback device as described above, and there has been no method for realizing such a request on the provider side.

  The present invention has been made in view of the above-described problems, and is a video composition that makes it possible to obtain a playback mode that reflects the intention of the provider with respect to a playback mode at the time of special playback of picture-in-picture video. A device or the like is provided.

In order to solve the above-described problem, a video composition device according to the present invention receives a plurality of video data and management data related to at least one of the plurality of video data, and combines the videos of the plurality of video data. A video composition device that outputs in-picture format playback video,
The management data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
Reading means for reading the instruction information from the management data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
The instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the main-screen video in the picture-in-picture playback video.

A video composition device of the present invention is a video composition device that outputs a plurality of video data and a reproduced video in a picture-in-picture format obtained by synthesizing the video of the plurality of video data.
Including at least one of the plurality of video data instruction information for instructing a playback mode at the time of special playback of the playback video;
Reading means for reading the instruction information from the video data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
The instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the main-screen video in the picture-in-picture playback video.

  In the video composition device of the present invention, the instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the video of the main screen in the picture-in-picture format. Instead, in the picture-in-picture playback video, information indicating whether or not the video on the child screen is specially played back while the video on the parent screen is specially played back is provided.

  In the video composition device of the present invention, the instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the video of the main screen in the picture-in-picture format. Instead, in the picture-in-picture playback video, information indicating whether or not the video on the parent screen is specially played back while the video on the child screen is specially played back is provided.

The program of the present invention reads a plurality of video data and management data related to at least one of the plurality of video data, and outputs a playback video in a picture-in-picture format obtained by combining the videos of the plurality of video data A program executed in the synthesis apparatus,
The management data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
The video composition device
Reading means for reading the instruction information from the management data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
In accordance with the instruction information, in the picture-in-picture playback video, a first control for performing control to instruct the combining means whether or not the video on the child screen is displayed during the special playback of the video on the parent screen The step is realized.

The program of the present invention is a program that is executed in a video synthesizing apparatus that reads a plurality of video data and outputs a playback video in a picture-in-picture format in which the videos of the plurality of video data are combined.
Including at least one of the plurality of video data instruction information for instructing a playback mode at the time of special playback of the playback video;
The video composition device
Reading means for reading the instruction information from the at least one video data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
In accordance with the instruction information, in the picture-in-picture playback video, a first control for performing control to instruct the combining means whether or not the video on the child screen is displayed during the special playback of the video on the parent screen The step is realized.

  Further, the program of the present invention may replace the first control step with the picture-in-picture playback video, whether the video on the child screen is specially played back while the video on the parent screen is specially played back. A second control step for performing control to instruct the combining means is realized.

  In addition, the program of the present invention may replace the first control step with the picture-in-picture playback video, whether the video on the parent screen is specially played back while the video on the child screen is specially played back. A third control step for performing control to instruct the combining means is realized.

The program of the present invention reads a plurality of video data and management data related to at least one of the plurality of video data, and outputs a playback video in a picture-in-picture format obtained by combining the videos of the plurality of video data A program executed in the synthesis apparatus,
The management data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
The video composition device includes a reading unit that reads the instruction information from the management data,
The program includes a synthesizing step of synthesizing videos of the plurality of video data in the instructed playback mode according to the instruction information.
The instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the main-screen video in the picture-in-picture playback video.

The program of the present invention is a program executed in a video composition device that outputs a plurality of video data and a reproduced video in a picture-in-picture format obtained by synthesizing the video of the plurality of video data.
Including at least one of the plurality of video data instruction information for instructing a playback mode at the time of special playback of the playback video;
The video composition device includes a reading unit that reads the instruction information from the at least one video data,
The program includes a synthesizing step of synthesizing videos of the plurality of video data in the instructed playback mode according to the instruction information.
The instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the main-screen video in the picture-in-picture playback video.

  Further, in the program of the present invention, the instruction information is replaced with information indicating whether or not a sub-screen video is displayed during special playback of the picture-in-picture video. In the picture-in-picture playback video, the information indicating whether or not the video on the child screen is specially played while the video on the parent screen is specially played back.

  Further, in the program of the present invention, the instruction information is replaced with information indicating whether or not a sub-screen video is displayed during special playback of the picture-in-picture video. In the picture-in-picture playback video, the information indicating whether or not the video on the parent screen is specially played while the video on the child screen is specially played back.

  In the present invention, the instruction flag indicating the reproduction mode of the sub-screen video (whether or not to follow the special playback of the main screen) at the time of special playback of the main-screen video is included in the video data of the sub-screen video. include. Alternatively, it is stored in management data independent of the video data of the parent screen and the child screen. Thus, the instruction information by the instruction flag is handled together with the video data when the video data is distributed. In the video composition device, the stored instruction flag is read and used to perform special playback at the time of picture-in-picture as intended by the provider (the sub-screen video follows the parent screen video and is specially played. Playback or playback independently). As a result, the playback mode at the time of picture-in-picture becomes independent of the implementation of the video composition device and playback equipment.

1 is a functional block diagram showing a schematic configuration of a video reproduction apparatus according to a first embodiment. It is explanatory drawing which showed the reproduction | regeneration form at the time of special reproduction | regeneration with the video reproduction apparatus concerning 1st Embodiment. It is the figure which showed another reproduction | regeneration form at the time of special reproduction | regeneration with the video reproduction apparatus concerning 1st Embodiment. It is explanatory drawing which showed another reproduction | regeneration form at the time of special reproduction | regeneration with the video reproduction apparatus concerning 1st Embodiment. It is explanatory drawing which showed another reproduction | regeneration form at the time of special reproduction | regeneration with the video reproduction apparatus concerning 1st Embodiment. It is the table | surface which showed the correspondence of the value of the instruction | indication flag handled with the video reproduction apparatus concerning 1st Embodiment, and the reproduction | regeneration form at the time of special reproduction | regeneration. 1 is a functional block diagram illustrating a schematic configuration of a video data generation device according to a first embodiment. It is explanatory drawing which showed schematic structure of the video data handled by 1st Embodiment. It is a functional block diagram showing another composition of a video reproduction device concerning a 1st embodiment. It is explanatory drawing which showed another schematic structure of the video data handled by 1st Embodiment. FIG. 5 is a functional block diagram showing still another configuration of the video reproduction device according to the first embodiment. It is explanatory drawing which showed the example of a screen display of the image | video reproduced | regenerated by 1st Embodiment. It is the functional block diagram which showed schematic structure of the video reproduction apparatus concerning 2nd Embodiment. It is the functional block diagram which showed schematic structure of the video reproduction apparatus concerning 3rd Embodiment. It is the functional block diagram which showed schematic structure of the video data generation apparatus concerning 3rd Embodiment. It is explanatory drawing which showed schematic structure of the video data handled by 3rd Embodiment. It is explanatory drawing which showed the implementation | achievement system of the conventional picture in picture function. It is an example showing a conventional picture-in-picture function. It is explanatory drawing explaining the delivery of the conventional video content.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In the following embodiments, as an example, a case where the video composition device (program) of the present invention is applied to a video playback device that plays back a recording medium will be described.

(First embodiment)
First, the video playback apparatus according to the first embodiment will be described with reference to FIGS.

  FIG. 1 is a functional block diagram showing a schematic configuration of a video reproduction apparatus 1 according to the first embodiment. The video reproduction apparatus 1 receives two pieces of video data (encoded video stream), decodes them, and outputs and displays a video obtained by synthesizing the decoded video. In the present specification, the video displayed on the main screen in picture-in-picture display is referred to as “main video”, and the video displayed as a sub-screen is referred to as “sub video”.

  The video reproduction apparatus 1 holds a flag reading unit 101 for reading out an instruction flag, which will be described later, from the video data of the sub video, decoding units 102 and 103 for decoding the encoded data for each video data, and also holds the respective decoded videos Buffers (BF) 104 and 105 for outputting in accordance with the synthesizing timing, a synthesizing unit 106 for synthesizing the decoded video, a display unit 107 for displaying the output video, an input unit 108 for receiving input from the user, In addition, a decoding management unit 109 for controlling the decoding processing of the decoding units 102 and 103 is provided. The synthesizing unit 106 includes an adjusting unit 110 for adjusting a display size and a display position for synthesizing the decoded video of the sub video.

  In the video reproduction apparatus 1 according to the present embodiment, an instruction flag (to be described later) is read from the video data and output by the flag reading unit 101. In actual video data decoding, the accompanying audio data is separated, management information accompanying the video (information necessary for decoding data such as information indicating the video encoding method, and temporal extraction and connection of video) In some cases, it is necessary to read out information such as play list information to be specified, which is necessary for reproducing the video, but in FIG.

  First, the flag reading unit 101 reads an instruction flag that indicates a playback mode at the time of special playback from the video data of the sub video. The read instruction flag is input to the decoding management unit 109 and used during special playback of picture-in-picture video. Specific contents of the instruction flag will be described in detail later.

  First, the operation during normal playback will be described. In the case of normal playback, the video data of the main video and the video data of the sub video (after reading the instruction flag) are input to the decoding units 102 and 103, respectively, and sequentially decoded. Each decoded video is temporarily stored in the buffers (BF) 104 and 105 and then sent to the synthesis unit 106 after adjusting the output timing.

  The combining unit 106 includes an adjustment unit 110 inside. Of the two decoded videos input to the synthesis unit 106, the decoded video corresponding to the sub video is input from the BF 105 to the adjustment unit 110. The adjustment unit 110 changes the image size of the sub video and sets the display position as preprocessing for superimposing the sub video on the main video. The values of the image size and display position of these sub-videos (child screens) are previously stored in the video playback device 1 as synthesis parameters, or input by the user. Alternatively, it may be determined through interaction with the user. The sub video adjusted in this way is superimposed on the decoded video corresponding to the main video input from the BF 104 and synthesized.

  Note that when compositing, the transparency of the sub video may be set so that the main video can be seen through. This transparency is also previously stored in the video reproduction apparatus 1 as a synthesis parameter, or is determined through interaction with the user.

  The picture-in-picture video synthesized and output by the synthesis unit 106 is displayed on a monitor or the like through the display unit 107.

  Next, the operation when special playback is instructed will be described. The video playback device 1 includes an input unit 108 and accepts a special playback instruction from the user. Upon receiving the special reproduction instruction input, the input unit 108 converts it into special reproduction instruction information indicating the content of the instruction, and sends the special reproduction instruction information to the decoding management unit 109.

  In addition to the special reproduction instruction information from the input unit 108, the decoding management unit 109 receives an instruction flag indicating the reproduction mode at the time of special reproduction read from the video data of the sub video by the flag reading unit 101. When the decoding management unit 109 receives the special reproduction instruction information, the decoding management unit 109 controls the processing of the two decoding units 102 and 103 and the synthesizing unit 106 so that the picture can be played in one of the reproduction modes as shown in FIGS. -Perform special playback during in-picture. At this time, the instruction flag input from the flag reading unit 101 is used to determine which playback mode is used for special playback.

  The playback mode (A) shown in FIG. 2 shows a playback mode in which the sub video is also specially played following the main video when the special playback of the main video is instructed. At this time, the decoding management unit 109 of the video reproduction apparatus 1 instructs both the decoding units 102 and 103 to perform special reproduction (execution of the decoding process) in synchronization. Upon receiving the instruction, for example, in the case of high-speed playback, the decoding units 102 and 103 realize high-speed playback of both the main video and the sub-video by thinning out the frame data to be decoded, for example, by decoding and outputting only the I frame. The output decoded video is input to the synthesizing unit 106 in the same manner as in normal reproduction, and is synthesized, output, and displayed.

  In order to facilitate special reproduction in synchronism in this way, a process of matching the insertion position of the I frame between the main video and the sub video at the time of encoding may be added. Furthermore, the data structure of the main video and the sub video may be matched, including the GoP (Group of Picture) structure which is the data structure of the inter-frame predictive coding system defined by the MPEG (Moving Picture Expert Group) standard. Good. By combining the data structure of the main video and the sub video including the GoP structure, it is possible to synchronize with smooth special reproduction including the P frame.

  In the playback mode (B) shown in FIG. 3, when the special playback of the main video is instructed, the output of the sub video is stopped during the special playback of the main video, and the normal playback is resumed after returning to the normal playback. It shows a reproduction form in which sub-videos from the same time position as the main video are synthesized again and displayed. At this time, the decoding management unit 109 of the video reproduction apparatus 1 instructs only the decoding unit 102 to perform special reproduction (execution of the decoding process). In addition, the decoding management unit 109 instructs the synthesis unit 106 to stop the synthesis of the sub video during special playback. When the return to normal playback is instructed, the decoding management unit 109 instructs the decoding unit 102 to perform normal playback (execution of the decoding process), and the decoding unit 103 is the same as the main video that has returned to normal playback. Decodes the video data of the sub video from the time position. After the normal reproduction is restored, the main and sub decoded images are input to the combining unit 106, combined, output, and displayed.

  2 and 3 are examples in which the main video and the sub video are played back in synchronization. That is, in the playback modes of FIGS. 2 and 3, the main video and the sub video are played based on the same time reference.

  The playback mode (C) shown in FIG. 4 shows a playback mode in which when the special playback of the main video is instructed, only the main video is played back specially and the sub video is played back and displayed as normal playback. . At this time, the decoding management unit 109 of the video reproduction apparatus 1 instructs only the decoding unit 102 to perform special reproduction (execution of the decoding process). The decoding unit 103 performs decoding while performing normal reproduction. Then, the specially reproduced main video output from the decoding unit 102 and the normally reproduced sub-video output from the decoding unit 103 are input to the synthesizing unit 106 and synthesized as in the normal reproduction. Output and display.

  In the playback mode (D) shown in FIG. 5, when the special playback of the main video is instructed, the output of the sub video during the special playback of the main video is stopped, and after returning to the normal playback, 3 shows a playback mode in which the sub-videos are synthesized again and displayed. At this time, the decoding management unit 109 of the video reproduction apparatus 1 instructs only the decoding unit 102 to perform special reproduction (execution of the decoding process). In addition, the decoding management unit 109 instructs the synthesis unit 106 to stop the synthesis of the sub video during special playback. When the return to normal playback is instructed, the decoding management unit 109 instructs the decoding unit 102 to perform normal playback (execution of the decoding process), and the special playback is started to the decoding unit 103 so that the sub-video is reproduced. Instructs the decoding of the sub video from the position where the decoding is stopped. After the normal playback is restored, the main video and the sub video are input to the combining unit 106, combined, output, and displayed.

  4 and 5 are examples in which the playback of the main video and the sub video is asynchronous when special playback is performed. That is, in the playback modes of FIGS. 4 and 5, the main video and the sub video are played back with different time references.

  Here, the instruction flag will be described. The instruction flag read by the flag reading unit 101 and input to the decoding management unit 109 is used to determine which playback mode is used for special playback. That is, whether the sub video is synchronized with the main video (FIGS. 2 and 3) or asynchronous (FIGS. 4 and 5) is controlled according to the value of the instruction flag included in the video data of the sub video. In the present specification, the following three types are defined as examples of instruction flags to be set.

"Reproduction form instruction flag":
A binary flag indicating whether the main video is to be specially reproduced by following the sub-video, or is independently reproduced (for example, normal reproduction) without being followed.

“Follow-up playback prohibition flag”:
A binary flag that prohibits sub-pictures from following special playback when the main video is special playback. When this flag is “true”, the sub video is always played back independently of the main video special playback (eg, normal playback).

“Independent playback prohibition flag”:
A binary flag that prohibits sub-pictures from being played independently (eg, normal play) without following the main picture when the main picture is specially played. When this flag is “true”, the sub video is always specially played back following the main video.

  The video data includes any one or more of these instruction flags, or their values determine the playback mode at the time of special playback in picture-in-picture in the video playback device 1. The table in FIG. 6 shows specific correspondences between the values of the instruction flags and the possible reproduction modes (A) (FIG. 2) and reproduction modes (C) (FIG. 4).

  The table 601 in FIG. 6 shows the relationship between the values “independent” and “follow” of the playback mode instruction flag and the playback modes that the video playback apparatus 1 can take when the playback mode instruction flag is “independent” or “follow”. Yes. When the playback mode instruction flag is “independent”, the sub video does not follow the special playback of the main video and is normally played back on an independent time basis. When the reproduction form instruction flag is “follow”, the sub video follows the special reproduction of the main video and is specially reproduced based on the same time reference.

  In addition, in order to make it easy to perform special reproduction synchronously, when processing such as inserting the insertion position of the I frame in the main video and the sub video at the time of encoding is added, the value of this flag is directly set to I It can also be understood that it indicates whether the frame positions are matched (flag value is “following”) or not (flag value is “independent”).

  A table 602 in FIG. 6 shows the relationship between the values “true” and “false” of the follow-up reproduction prohibition flag and the reproduction modes that the video reproduction apparatus 1 can take when the follow-up reproduction prohibition flag is “true” or “false”. Yes. When the follow-up reproduction prohibition flag is “true”, the sub-picture does not follow the special reproduction of the main picture and is normally reproduced on an independent time basis. On the other hand, when the follow-up reproduction prohibition flag is “false”, the video reproduction apparatus 1 may perform any reproduction.

  Table 603 in FIG. 6 shows the relationship between the independent reproduction prohibition flag values “true” and “false” and the reproduction modes that the video reproduction apparatus 1 can adopt when the independent reproduction prohibition flag is “true” or “false”. Is shown. When the independent reproduction prohibition flag is “true”, the sub video follows the special reproduction of the main video and is specially reproduced on the same time basis. On the other hand, when the independent reproduction prohibition flag is “false”, the video reproduction apparatus 1 may perform any reproduction.

  The instruction flags shown here are only examples, and both are binary flags of “0” and “1”. However, it is also possible to define and use more complicated instruction flags consisting of multiple values. For example, it is a flag that prohibits follow-up playback and allows independent playback, and independent playback in a format that stops playback of sub-video as shown in FIG. 5 is prohibited, and sub-video is normally played back as shown in FIG. It is also possible to provide an instruction flag such that only independent playback of the continued format is permitted.

  In other words, in addition to the binary flags of “0” and “1” indicating whether the playback is “following” or “independent”, whether the main video continues to display the sub video during special playback (FIGS. 2 and 2). 4) or “0” and “1” binary flags indicating “display” or “non-display” indicating whether the sub video is not displayed when the main video is special playback (FIGS. 3 and 5). It is also possible to uniquely specify one of the reproduction modes (A) to (D) in FIGS. 2 to 5 by combining these flag values. The former (flag indicating whether playback is “following” or “independent”) is called “reproduction instruction flag”, and the latter (flag indicating whether sub-video is “display” or “non-display”) is called “display instruction flag”. You may distinguish.

  For example, when the reproduction instruction flag is “follow” and the display instruction flag is “display”, the reproduction mode (A) in FIG. 2 is selected. In such a playback, the sub video combined with the main video can appear to be completely part of the main video, and has the effect of making the main video and the sub video indistinguishable. . Therefore, when the video provider desires such an effect, each instruction flag is set as described above. On the other hand, when it is desired to clearly indicate that the sub video is an additional video different from the main video, the playback instruction flag is set to “follow”, the display instruction flag is set to “non-display”, and the playback mode (B) in FIG. Choose.

  Further, when the reproduction instruction flag is “independent” and the display instruction flag is “display”, the reproduction mode (C) of FIG. 4 is selected. This is a playback mode applied to a sub video with a very high priority, for example, in which the actual playback time for one sub video playback is strictly determined. During this period, the sub video playback is not stopped. On the other hand, in the playback mode (D) of FIG. 5 in which the playback instruction flag is “independent” and the display instruction flag is “non-display”, the playback and display of the sub video are stopped while the main video is special playback, After the video returns to normal playback, the sub video that follows from the stop position is played back and displayed. For this reason, it is possible to avoid a situation in which the sub video is inadvertently missed while operating the device, and it is suitable for showing the sub video calmly.

  In this way, it is possible to specify the playback mode at the time of special playback in the video playback device 1 by the instruction flag given accompanying the video data. This instruction flag is added when the video data provider generates video data of the sub video.

  FIG. 7 shows a functional block diagram illustrating a schematic configuration of the video data generation device 11. The video data generation device 11 includes an encoding unit 701, a flag setting unit 702, and a flag writing unit 703. The instruction input from the video provider is converted into an instruction flag by the flag setting unit 702. The input video is encoded by the encoding unit 701 and converted into video data (encoded video stream). Thereafter, an instruction flag is stored in the video data (encoded video stream) through the flag writing unit 703.

  By embedding the instruction flag in the video data (encoded video stream), the video provider enables special playback of picture-in-picture on the video playback device in the playback mode intended by the provider. FIG. 8 shows an example of video data generated by the video data generation device 11 of FIG. 7, and particularly shows an example of an embedding position of the instruction flag. As shown in FIG. 8, the instruction flag is embedded in the sequence header in the video data (encoded video stream) (801), embedded in each packet header (802), or management information for the encoded video stream ( If play list information or the like is recorded in the video data, an instruction flag is stored in the management information (803). However, the embedding position of the instruction flag in the video data is not limited to this.

  In the video reproduction apparatus 1 according to the first embodiment described above, the example in which the instruction flag is included in the input video data (encoded video stream) is shown, but the present invention is not limited to this. For example, together with the video data stream, management information for managing the video data stream (information necessary for decoding data such as information indicating the video encoding method, and play specifying the temporal cutout and connection of video) When a data stream (management data stream) in which recorded information (list information such as list information) is recorded is provided and multiplexed data obtained by multiplexing these data streams is input, it is not included in the video data stream but in the management data stream. The instruction flag may be stored and input.

  Further, in the video data input to the video playback device 1 described above, an example in which the instruction flag is included in the video data of the sub video is shown. However, the present invention is not limited thereto, and the instruction flag is set on the video data side of the main video. It is also assumed that the data is stored and input. The instruction flag in this case indicates whether or not any sub video follows the special reproduction of the main video when video data of any sub video is combined with the video data of the main video. It becomes a flag.

  FIG. 9 shows the differences between these input patterns. Reference numerals 91 and 92 denote cases where two pieces of video data (encoded video stream) including an instruction flag are input. 91 is the same as α in FIG. Reference numerals 93 and 94 denote cases where multiplexed data obtained by multiplexing management data (stream) including instruction flags and video data (stream) is input. Separators 903 and 904 are functional units for separating management data from the multiplexed data. Note that the description of the handling of the separated management data is omitted here. Reference numerals 91 and 93 indicate that the instruction flag is included in the video data (or corresponding management data) on the sub-video side, and reference numerals 92 and 94 indicate that the instruction flag is included in the video data (or corresponding management data) on the main video side. Shows the case. In either case, the output is two pieces of video data and an instruction flag, which are connected in place of the α portion in FIG.

  In FIG. 9, 93 and 94, the multiplexed data is depicted as being composed of only two data streams of video data and management data, but in reality, other data streams such as audio data streams are included. It may be multiplexed. In FIG. 9, these are not shown for convenience.

  FIG. 10 is a diagram illustrating an example of video data (encoded video stream) and management data stream constituting multiplexed data. As shown in FIG. 10, the instruction flag is embedded in reproduction management information (playlist information or the like) relating to video data (encoded video stream) (1001), or decoding management information relating to video data (encoded video stream). (1002). However, the embedding position in the management data of the instruction flag is not limited to this.

  The two decoding units 102 and 103 in FIG. 1 may have the same decoding capability, or may have different decoding capabilities. In the video playback device 1 according to the first embodiment, at the time of input, each video data is determined as video data for main video or video data for sub video. Therefore, it is possible to define video data dedicated to sub video having a resolution lower than that of the main video (image size is small), and in such a case, the decoding unit 103 to which the video data of the sub video is input is input to the sub video. A dedicated one for decoding low-resolution video can also be used for decoding dedicated video data.

  In the video reproduction device 1 of FIG. 1, a case may be assumed in which the input video data is unencoded video data. In this case, since decoding of the video data is not necessary, the decoding units 102 and 103 are not functional blocks that perform decoding, but function blocks (separating units 102 ′ and 103 ′) that appropriately input and output the input video data. Replaced. Also, the decoding management unit 109 is replaced with a functional block (separation management unit 109 ') for managing that the functional blocks (separation units 102' and 103 ') separate and output video data.

  Further, the synthesizing unit 106 of the video reproduction apparatus 1 shown in FIG. 1 only adjusts the sub video and does not adjust the main video (that is, the main video is displayed in full screen) (FIG. 11A). However, the composition unit 106 is not limited to this, and as shown in FIG. 11B, an adjustment unit 1110 may be provided on the main video side, and replaced with a composition unit 1106 that performs adjustment and outputs. FIG. 12 shows an example of the output video when the composition unit 1106 is replaced. In this case, the main video is changed to an arbitrary size as shown at 1202 in FIG. 12, or the main and sub two videos are displayed side by side without overlapping each other as shown at 1203. A screen configuration can also be used.

  In addition to the picture-in-picture display application, the video playback apparatus of the present embodiment can be applied to a video playback apparatus that displays two screens as shown by 1203 in FIG. That is, by embedding the instruction flag in the video data, it is possible to control whether or not special playback of one video is followed by special playback of the other video. It is possible to perform special playback of the screen display video.

  In the video playback apparatus 1 according to the first embodiment described above, an instruction flag which is information indicating a playback mode at the time of special playback in picture-in-picture is input together with video data, and the special flag is input based on the instruction flag. A playback mode at the time of playback is determined. Therefore, when the video provider generates video data, an instruction flag is sent along with the video data, so that special playback at the time of picture-in-picture playback in the playback format intended by the provider is sent to the video playback device. Can be done.

  In the case of the video playback apparatus 1 according to the present embodiment, 91 in FIG. 9, it is determined whether each video data is for main video or sub video at the time of input, but it is generated for sub video. The recorded video data can be input as the video data of the main video. In that case, the instruction flag stored in the video data is ignored. Similarly, in the case of 92 in FIG. 9, the video data including the instruction flag generated for the main video can be input as the video data of the sub video. Even in this case, the instruction flag stored in the video data is ignored. Further, a default playback mode may be provided in the video playback device 1 so that special playback is performed in the default playback mode when the instruction flag is not read from either the main video or the sub video. .

  The instruction flag described in the first embodiment is a flag indicating the behavior of the sub video when the main video is specially played back. Conversely, when the special playback of the sub video is instructed, An instruction flag indicating the behavior of the main video can also be defined. This is not particularly noted in the following embodiments, but the same applies to other embodiments.

  In addition to the instruction flag indicating the playback mode at the time of special playback, a flag for prohibiting movement / enlargement of the sub video is also stored as a flag indicating the behavior of the video and notifying the intention of the video provider. You may make it do. The flag prohibiting the movement / enlargement of the sub video is applied to a playback apparatus in which the display size and display position of the sub video can be changed by the user. When the value of the flag prohibiting the enlargement / movement of the sub video is “true”, the display size and the display position of the sub video are fixed with respect to the main video, and the user instruction is not accepted.

  In the video playback device 1 of the first embodiment, the method for controlling the processing of the decoding unit is shown as a method for realizing special playback. However, the method for realizing special playback is not limited to this. For example, in the example of high-speed playback, in addition to the process of thinning out by the decoding unit shown in the above embodiment, after decoding all the frames, the decoded images output from the buffers 104 and 105 are thinned out together with the timing adjustment. Such a method can also realize high-speed playback. That is, the video playback device 1 shown in FIG. 1 is a configuration form of a device for realizing special playback, and other device configurations for realizing special playback can be adopted. The special reproduction control by the instruction flag of the embodiment can be applied. This is not particularly noted in the following embodiments, but the same applies to other embodiments.

(Second Embodiment)
Next, the video reproduction apparatus 2 according to the second embodiment will be described with reference to FIG.

  FIG. 13 is a functional block diagram showing a schematic configuration of the video reproduction device 2 according to the second embodiment. The video playback device 2 includes flag reading units 1301 and 1302 for reading instruction flags from each video data, decoding units 1303 and 1304, buffers (BF) 1305 and 1306, a synthesis unit 1307 (including adjustment units 1312 and 1313), The display unit 1308, the input unit 1309, the decryption management unit 1310, and the composition management unit 1311 for controlling the composition processing executed by the composition unit 1307 are configured. Of these, the flag reading units 1301 and 1302 all have the same functions as the flag reading unit 101 shown in FIG. 1, and the decoding units 1303 and 1304, buffers 1305 and 1306, and the display unit 1308 are shown in FIG. The decoding units 102 and 103, the buffers 104 and 105, and the display unit 107 shown in FIG. Furthermore, the adjustment units 1312 and 1313 have functions equivalent to those of the adjustment unit 110 illustrated in FIG. Hereinafter, the operation of the video reproduction device 2 according to the second embodiment will be described focusing on differences from the video reproduction device 1 according to the first embodiment.

  In the video reproduction apparatus 1 according to the first embodiment described above, two input video data are input after being preliminarily determined which is for main video and which is for sub video at the stage of input. It was. This is most effective when there is video data that does not reproduce the main video, such as video data dedicated to the sub video, for example, a bonus video for the main video or a short commentary video of the shooting secret story.

  On the other hand, the video playback device 2 according to the second embodiment does not divide the two input video data into main and sub ones at the input stage, and inputs two videos at the video synthesis stage after input. The main video and the sub video are determined by adjusting the synthesis of the video. This is most effective in applications in which main video reproduction and sub-video reproduction are frequently switched, for example, in the case where display video is frequently switched in multi-angle display.

  For this purpose, the video playback device 2 has a new functional block and composition management unit 1311 that are not present in the video playback device 1.

  In addition to the special playback instruction, the input unit 1309 accepts a video selection instruction from the user to switch between the main video and the sub video (select which is the main video and which is the sub video). Upon receiving the video selection instruction, the input unit 1309 issues video selection instruction information and transmits it to the composition management unit 1311.

  The composition management unit 1311 knows which is the main display and which is the sub-display in the video selection instruction information, and sends the main video indicated by the video selection instruction information to each of the adjustment units 1312 and 1313. Composite parameters corresponding to display and sub-video display are sent to each. In addition, the composition unit 1307 is notified of the vertical relationship (superimposing the sub video on the main video) when overlapping. Further, it may be sent together with the setting of transparency. Further, selection information indicating which is selected as the sub video is sent to the decoding management unit 1310.

  The flag reading units 1301 and 1302 read instruction flags from the input video data, and any of the read instruction flags is input to the decoding management unit 1310.

  When receiving the special reproduction instruction from the user and receiving the special reproduction instruction information issued by the input unit 1309, the decoding management unit 1310 receives the selection information received from the composition management unit 1311 (which is selected as the sub-video). As shown in the first embodiment, the instruction flag of the video data corresponding to the sub video is obtained from the two input instruction flags, and the instruction flag is used to determine the instruction flag of the video data. The reproduction mode of special reproduction at the time of picture-in-picture reproduction is determined.

  In the video playback device 2, the two input video data are video data that can be played back as a main video and a sub video. Therefore, in the video reproduction device 2, the decoding units 1303 and 1304 have completely equivalent functions.

(Third embodiment)
Next, the video reproduction apparatus 3 according to the third embodiment will be described with reference to FIGS. 14 to 16.

  FIG. 14 is a functional block diagram showing a schematic configuration of the video reproduction apparatus 3 according to the third embodiment. The video reproduction device 3 includes a separation unit 1401 for separating multiplexed data obtained by multiplexing a plurality of video data into a plurality of video data and management data, a flag reading unit 1402 for reading an instruction flag from the management data, Decoding units 1403 and 1404, two buffers (BF) 1405 and 1406, a combining unit 1407 (including the adjusting unit 1411), a display unit 1408, an input unit 1409, and a decoding management unit 1410 are configured. Among these, the decoding units 1403 and 1404, the buffers 1405 and 1406, the synthesis unit 1407, the adjustment unit 1411, the display unit 1408, and the decoding management unit 1410 are respectively the decoding units 102 and 103 of the video reproduction apparatus 1 illustrated in FIG. 104, 105, the synthesis unit 106, the adjustment unit 110, the display unit 107, and the decryption management unit 109. That is, the video playback device 3 according to the third embodiment is different from the video playback device 1 according to the first embodiment in that the separation unit 1401 and the flag reading unit 1402 (the flag reading unit 101 in the video playback device 1), Only the input unit 1409 (the input unit 108 in the video reproduction apparatus 1) is different. Hereinafter, the operation of the video reproduction device 3 according to the third embodiment will be described focusing on differences from the video reproduction devices 1 and 2 according to the first and second embodiments.

  In the video reproduction apparatuses 1 and 2 described above, the instruction flag is stored in the video data (encoded video stream) or the management data stream corresponding to the video data on a one-to-one basis. For video data, the playback mode at the time of special playback when the video data is displayed as a sub-video by picture-in-picture playback is controlled. In contrast, the video playback apparatus 3 according to the third embodiment manages a plurality of video data (encoded video streams) in a situation where a management data stream for managing them is independently given. The playback mode at the time of special playback in picture-in-picture playback is controlled by the instruction flag stored in the data stream.

  That is, the instruction flag handled by the video playback device 3 does not indicate the playback mode for any of the input video data, but is determined after one is determined as main video data and the other as sub video data. It operates to control the playback mode of the sub-video. For this reason, the video playback device 1 inputs the video data of the main video and the sub video in advance after the determination, and the video playback device 2 determines whether the main video or the sub video is determined at the synthesis stage, while the video playback In the apparatus 3, when the multiplexed data is separated by the separation unit 1401, which is the main video and which is the sub video is determined, and the processing after the decoding units 1403 and 1404 is taken over.

  The video playback apparatus 3 according to the present embodiment inputs two video data (encoded video stream) and multiplexed data obtained by multiplexing a management data stream including an instruction flag, and synthesizes decoded images of the two video data. It is displayed as an image. FIG. 15 is a functional block diagram showing a schematic configuration of the video data generation device 31 for generating the input multiplexed data.

  The video data generation device 31 in FIG. 15 includes two encoding units 1501 and 1502, a management information generation unit 1503, a flag setting unit 1504, a flag writing unit 1505, and a multiplexing unit 1506.

  The two input videos are converted into video data (encoded video stream) by encoding units 1501 and 1502, respectively. The flag setting unit 1504 generates an instruction flag corresponding to the playback format instructed by the video provider. The generated instruction flag is the flag writing unit 1505 and the management information (management data generated by the management information generating unit 1503). Stream).

  The management information stored in the management data stream includes information (decoding management information) recorded corresponding to each video data, such as information indicating the encoding method of video data (encoded video stream), and temporal information of video. Common information (playback management information) such as playlist information that designates playback of a plurality of video data based on the cut-out and link information.

  FIG. 16 shows an example of data before multiplexing of data generated by the video data generation device 31 of FIG. 15 (an example of an instruction flag embedding position). As shown in FIG. 16, the instruction flag is stored in common reproduction management information of the management data stream (1601). These video data and management data are multiplexed by a multiplexing unit 1506 to form multiplexed data.

  After the multiplexed data generated by the video data generation device 31 shown in FIG. 15 is input, the operation of the separation unit 1401 of the video reproduction device 3 of FIG. 14 is as follows.

  Similar to the video playback device 2 of the second embodiment, the input unit 1409 accepts a video selection instruction from the user to switch between the main video and the sub video (select which is the main video and which is the sub video). . Upon receiving the video selection instruction, the input unit 1409 issues corresponding video selection instruction information and transmits it to the separation unit 1401.

  The separation unit 1401 separates the input multiplexed data into two pieces of video data and management data, and then, according to the received video selection instruction information, the video data designated as the main video to the decoding unit 1403 and the sub video. The designated video data is input to the decoding unit 1404. Also, the management data output from the separation unit 1401 is input to the flag reading unit 1402 and the instruction flag is read out. The read instruction flag is input to the decoding management unit 1410 to control the playback mode at the time of special playback in picture-in-picture playback. Thereafter, the operation is the same as that described in the first embodiment.

  In FIGS. 14 to 16 of the third embodiment, it is described that the multiplexed data consists only of video data (encoded video stream) and management data (management data stream). Other data such as audio data may be multiplexed. Here, for convenience of explanation, illustration of data other than such video data and management data is omitted.

  Although the video data (encoded data stream) and the management data (management data stream) input to the video playback device 3 are illustrated as being multiplexed, they are not necessarily multiplexed. Two video data and one management data may be input through separate channels and processed. In other words, the video reproduction device 3 according to the third embodiment has a point that an instruction flag that does not depend on any video data and can be handled in common for a plurality of video data is given to the device through management data. Yes, not otherwise.

  In each of the above-described embodiments, the video according to each embodiment is assumed assuming that video data (and management data) input via a transmission path such as broadcasting or communication is directly played back through the video playback device. Although the playback apparatus has been described, the situation to which the present invention is applicable is not limited to this. For example, video data (and management data) is recorded in a recording medium in advance, and video data (and management) recorded in the recording medium is recorded. The present invention can be similarly applied to a situation in which (data) is sequentially read and reproduced. This is the case in the package distribution of FIG. The same applies to the situation in which the recorded video data (and management data) is read and reproduced after being once recorded on the recording medium via the transmission path. That is, the present invention is directed to a video data including the instruction flag described in detail in each embodiment or a recording medium on which management data is recorded, and to a video reproducing apparatus that reproduces such a recording medium. Is also applicable.

  In addition, the instruction flag stored in the video data (or management data) described in each embodiment is a fixed value for reflecting the intention of the video provider regarding the playback mode at the time of special playback of picture-in-picture video. It was described as information. However, the instruction flag stored in the video data (or management data) can also be used as a field for recording the user's playback preference on the playback side. Alternatively, the instruction flag may be rewritten on the video playback device side to a value limited by the playback capability of the video playback device. Such rewriting is performed when recording video data (and management data) input through a transmission path such as broadcasting or communication on a recording medium, or sequentially reading and reproducing video data (and management data) from the recording medium. At this time, the video reproduction device rewrites the value of the storage field of the instruction flag. This rewriting process may be input directly by the user or may be dynamically changed by a program such as Java (registered trademark). That is, the present invention can be applied to video data including the instruction flag described in detail in each embodiment or a management data video reproducing device regardless of where the value of the instruction flag is finally set. It is.

  It should be understood that the embodiments disclosed herein are illustrative and non-restrictive in every respect. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 1 Video reproduction apparatus 101 Flag reading part 102, 103 Decoding part 104, 105 Buffer (BF)
106 Combining Unit 107 Display Unit 108 Input Unit 109 Decoding Management Unit 110 Adjustment Unit

Claims (12)

A video synthesizing apparatus that receives a plurality of video data and management data related to at least one of the plurality of video data, and outputs a reproduced video in a picture-in-picture format by synthesizing the videos of the plurality of video data. ,
The management data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
Reading means for reading the instruction information from the management data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
The instruction information is the picture-in-picture format playback video, and is information indicating whether or not the sub-screen video is displayed during the special playback of the main-screen video. Synthesizer. A video synthesizing device that outputs a plurality of video data and a playback video in a picture-in-picture format obtained by synthesizing videos of the plurality of video data,
At least one of the plurality of video data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
Reading means for reading the instruction information from the video data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
The instruction information is the picture-in-picture format playback video, and is information indicating whether or not the sub-screen video is displayed during the special playback of the main-screen video. Synthesizer.   The instruction information is replaced with information indicating whether or not the sub-screen video is displayed during the special playback of the main-screen video in the picture-in-picture format. 3. The video composition apparatus according to claim 1 or 2, wherein the video synthesizer is information indicating whether or not the video on the child screen is specially reproduced while the video on the main screen is specially reproduced in the format of the reproduced video. .   The instruction information is replaced with information indicating whether or not the sub-screen video is displayed during the special playback of the main-screen video in the picture-in-picture format. 3. The video composition apparatus according to claim 1, wherein the video composition apparatus is information indicating whether or not the video on the parent screen is specially reproduced while the video on the child screen is specially reproduced in the format of the reproduced video. . Executed in a video composition device that reads a plurality of video data and management data related to at least one of the plurality of video data, and outputs a reproduced video in a picture-in-picture format that combines the videos of the plurality of video data. A program,
The management data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
The video composition device
Reading means for reading the instruction information from the management data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
In accordance with the instruction information, in the picture-in-picture playback video, a first control for performing control to instruct the combining means whether or not the video on the child screen is displayed during the special playback of the video on the parent screen A program for realizing steps. A program that is executed in a video synthesizing apparatus that reads a plurality of video data and outputs a playback video in a picture-in-picture format in which the videos of the plurality of video data are synthesized.
At least one of the plurality of video data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
The video composition device
Reading means for reading the instruction information from the at least one video data;
Synthesizing means for synthesizing videos of the plurality of video data in the instructed playback mode in accordance with the instruction information read by the reading means;
With
In accordance with the instruction information, in the picture-in-picture playback video, a first control for performing control to instruct the combining means whether or not the video on the child screen is displayed during the special playback of the video on the parent screen A program for realizing steps.   In place of the first control step, it is instructed to the synthesis means whether or not the picture on the child screen is specially reproduced while the picture on the main screen is specially reproduced in the picture-in-picture format. The program of Claim 5 or 6 for implement | achieving the 2nd control step which performs control.   In place of the first control step, instructing the combining means whether or not the parent screen image is specially reproduced while the child screen image is specially reproduced in the picture-in-picture reproduction image. The program of Claim 5 or 6 for implement | achieving the 3rd control step which performs control. Executed in a video composition device that reads a plurality of video data and management data related to at least one of the plurality of video data, and outputs a reproduced video in a picture-in-picture format that combines the videos of the plurality of video data. A program,
The management data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
The video composition device includes a reading unit that reads the instruction information from the management data,
The program includes a synthesizing step of synthesizing videos of the plurality of video data in the instructed playback mode according to the instruction information.
The instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the main-screen video in the picture-in-picture playback video. . A program that is executed in a video composition device that outputs a plurality of video data and a reproduced video in a picture-in-picture format in which the videos of the plurality of video data are synthesized,
At least one of the plurality of video data includes instruction information for instructing a playback mode at the time of special playback of the playback video,
The video composition device includes a reading unit that reads the instruction information from the at least one video data,
The program includes a synthesizing step of synthesizing videos of the plurality of video data in the instructed playback mode according to the instruction information.
The instruction information is information indicating whether or not a sub-screen video is displayed during special playback of the main-screen video in the picture-in-picture playback video. .   The instruction information is replaced with information indicating whether or not the sub-screen video is displayed during the special playback of the main-screen video in the picture-in-picture format. The program according to claim 9 or 10, wherein the program is information indicating whether or not the video of the child screen is specially reproduced while the video of the main screen is specially reproduced in the format of the reproduced video.   The instruction information is replaced with information indicating whether or not the sub-screen video is displayed during the special playback of the main-screen video in the picture-in-picture format. The program according to claim 9 or 10, wherein the program is information indicating whether or not the video on the main screen is specially reproduced while the video on the child screen is specially reproduced in the format of the reproduced video.

Images