JP2007158983A - Apparatus and method of reproduction, recording method, video display apparatus and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce processing load in realizing a picture-in-picture function. <P>SOLUTION: A second video image to be displayed as picture-in-picture has an SD size, and a region is divided to record a desired video image. Decoding is performed as the SD size, and the video image is displayed on each desired region depending on the position information of each region. Thus, processing amount can be reduced and a sufficient effect can be obtained as picture-in-picture. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a playback method, a playback device, and a video display device for playing back data from a recording medium, and more particularly to the implementation of a picture-in-picture function. The present invention also relates to a recording medium corresponding to the picture-in-picture function and a recording method for recording data on the recording medium.

Background art related to picture-in-picture is disclosed in Patent Document 1 below.
Specifically, in order to “realize a picture-in-picture function and a wallpaper display function in a BD-ROM” (issue of the summary of Patent Document 1), “planes 10, 11 and 12 for displaying moving images, subtitles, and graphics, respectively, respectively. In addition, a second video plane 50 for displaying a moving image is added, and outputs of the second video plane 50 and the video plane 10 are selected in units of pixels by a switch 51. Reduced moving image data is stored in the second video plane 50. By storing and switching the switch 51 in units of pixels corresponding to the display position of the reduced moving image data, the reduced moving image data of the second video plane 50 is displayed on the child screen with respect to the moving image data of the video plane 10. By storing wallpaper image data instead of moving image data in the video plane 10, the reduced moving image data Background wallpaper display screen as if displayed obtained "is disclosed that the (Patent solutions Summary of Document 1) as.

JP 2005-123775 A

  In Patent Document 1, more specifically, a picture-in-picture function is realized by using a plurality of video planes or a plurality of frame buffers and selecting a desired output for each pixel. In particular, when the focus is on video data, the configuration is shown in such a manner that two moving picture streams are decoded by a single MPEG video decoder as a player decoder for realizing them.

  Here, in consideration of the hardware configuration as an actual player, in order to decode two moving image streams with a single video decoder, it is necessary to process each in a time division manner. As a picture-in-picture function that also requires synchronous reproduction consideration, the amount of processing varies greatly depending on the encoding method and encoding rate of the moving picture stream to be decoded, the image size to be handled, and the like. In other words, whether or not the picture-in-picture function can be realized as a player depends on the processing capability of the decoder. In particular, in consideration of the playback environment on a PC, each process including video and audio decoding is part or Since it is conceivable that most of the processing is executed by software, the possible processing amount depends on the CPU capability such as the operating frequency.

  In Patent Document 1, the size of the video plane used for each of the video being reproduced (referred to as the first video) and the video displayed as the picture-in-picture on the sub-screen (referred to as the second video) is not different. Each image size was not limited. That is, no particular consideration has been given to the amount of processing when realizing the picture-in-picture function.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a playback device, a playback method, a recording method, a video display device, and a recording medium that take into account the reduction in processing load in realizing the picture-in-picture function. Is to provide.

  The above object is solved by the invention described in the claims.

  According to the playback device, playback method, recording method, video display device, and recording medium according to the present invention, the processing amount of the playback device can be reduced.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a block diagram of a reproducing apparatus according to the first embodiment of the present invention. 101 is an optical disk, 102 is a local storage, 103 is a PID filter, 104 is a video decoder 1, 105 is a video decoder 2, 106 is a graphics decoder 1, 107 is a graphics decoder 2, 108 a display image forming unit, 109 is an auxiliary information analysis unit , 110 is a reference time counter unit, 111 is a control unit, and 112 is a comparison unit.

  The optical disc 101 and local storage 102 store first video information and second video information, graphics information such as caption information and menu information, and auxiliary information associated therewith. In particular, which data is stored in the optical disk 101 or the local storage 102 is not limited, but in the present embodiment, in order to simplify the description, the first video information, caption information, menu information, and auxiliary information associated therewith are included. A case where the second video information and auxiliary information accompanying the second video information are stored in the local storage 102 on the optical disc 101 will be described. In this embodiment, both the first video information and the second video information are assumed to be HD size image data. Normally, audio information is multiplexed in the first video information and the second video information, but since the main point of the present invention is the video display method, the description will focus on the handling of the video information.

  The encoded data (stream) recorded on the optical disc 101 and the local storage 102 is packetized and recorded in a multiplexed state, for example, in the form of an MPEG2 system.

  First, the stream read from the optical disc 101 is distributed to the PID filter 103 according to the PID (packet ID). The moving image stream of the first video is input to the video decoder 1 (104). The video decoder 1 (104) decodes the input video stream, and the first video as a result is output to the display video forming unit. If the stream is subtitle information, the stream is input to the graphics decoder 1 (106). The graphics decoder 1 (106) performs necessary rendering in accordance with the input subtitle information stream, and the resulting graphics information is output to the display video forming unit. Similarly, when the stream input to the PID filter is menu information or the like, the stream is input to the graphics decoder 2 (107). In the graphics decoder 2 (107), necessary GUI parts and the like are prepared in accordance with the input stream of menu information and the like, and the resulting graphics information is output to the display video forming unit.

  The encoded data of the second video read from the local storage 102 is also distributed by the PID filter 103 and input to the video decoder 2 (105). In the video decoder 2 (105), the input encoded data is decoded, and the resulting second video is output to the display video forming unit.

  The reference time counter unit 110 counts the reference time for decoding, the video decoder 1 (104), the video decoder 2 (105), the graphics decoder 1 (106), the graphics decoder 2 (107), the display video forming unit 108, The reference time is output to the comparison unit 112.

  The video decoder 1 (104), the video decoder 2 (105), the graphics decoder 1 (106), and the graphics decoder 2 (107) perform a decoding process based on the reference time output from the reference time counter unit 110. Accordingly, it is possible to perform the decoding process by synchronizing the first video information and the second video information, as well as caption information, menu information, and the like.

Further, auxiliary information is read from the optical disc 101 or the local storage 102. In this embodiment, auxiliary information associated with the second video information is read from the local storage 102 from the viewpoint of realizing the picture-in-picture function. The read auxiliary information is sent to the auxiliary information analysis unit 109. The auxiliary information analysis unit 109 analyzes the auxiliary information and extracts the effective area information of the second video and the application time information thereof. The extracted effective area information is sent to the control unit 111, and the extracted application time information is sent to the comparison unit 112.
The comparison unit 112 compares the application time information with the reference time input from the reference time counter 110 and outputs the comparison result to the control unit 111.

  The control unit 111 uses the effective area information input from the auxiliary information analysis unit 109 to the display image forming unit 108 when the comparison result input from the comparison unit 112 indicates “match”. Gives instructions for displaying two images.

  The display video forming unit 108 synthesizes an output video screen based on the first video information, the second video information, and the graphics information. At that time, the second video information is synthesized based on the instruction content from the control unit 111.

Here, the operation of the display image forming unit 108 will be described in more detail with reference to FIG.
FIG. 2 is a diagram showing an example of the configuration of the display video forming unit 108. 201 is a first video plane, 202 is a second video plane, 203 is a presentation graphics plane, 204 is an interactive graphics plane, 205, 206, 208, 209, 211, and 212 are multipliers, and 207, 210, and 213 are adders. is there. The parts corresponding to those in FIG. 1 are denoted by the same reference numerals.

What is characteristic in this embodiment is that the effective area of the second video information stored in the second video plane 202 is limited, and the effective area information is separately input to the multipliers 205 and 206. Is a point. When these valid areas are not limited or valid area information is not applied, the configuration is basically the same as the conventional configuration. In order to show the feature points in the present embodiment, first, a basic display form of picture-in-picture also shown in the above-described conventional example will be described with reference to FIG.
The first video information output from the video decoder 1 (104) is stored in the first video plane 201. The second video information output from the video decoder 2 (105) is stored in the second video plane 202. Here, as shown in FIG. 3, the first video plane 201 and the second video plane 202 are both HD size, whereas the second video information has a resolution of HD size as a video, but the effective area is In this example, the video is pasted only on a part of the second video plane 202 in the SD size.

  The second video plane 202 is arranged in front of the first video plane 201 in the display order. At the time of video output, data is read out from each video plane in units of pixels, and the opacity α1 (1 or 0 in this case) is applied to the multipliers 205 and 206, whereby the display video 301 in FIG. 3 is output. Will be. That is, in this case, the opacity α1 = 1 within the rectangular area of the SD size stored in the second video plane 202, and α1 = 0 otherwise. After the outputs of the multipliers 205 and 206 are added by the adder 207, the opacity α2 is applied by the multipliers 208 and 209, and the adder 210 combines the subtitle information stored in the presentation graphics plane 203. Done. Further, the output of the adder 210 is applied with the opacity α3 by the multipliers 211 and 212, and the adder 213 is combined with the menu information stored in the interactive graphics plane 204, and the final display video is displayed. Is output. The synthesis of caption information and menu information has been described in the above-described conventional example, and an output example is not particularly illustrated in the present embodiment.

  Next, a case where an HD size video is simply used as the second video information will be described with reference to FIG. As shown in the second video plane 202 of FIG. 4, the second video information has a normal HD size. As for the output of the first video plane and the second video plane, the opacity α1 is evaluated over the entire area of the HD size, and the display video 401 is finally output. Regarding the determination of α1, all pixels whose luminance or color difference components are below (or above) a predetermined value are processed so that α1 = 0. This opacity determination method using pixel components is a common image composition method (for example, chroma key), and therefore, detailed description thereof is omitted.

  As described above, when an HD size video is used for the second video information, the video decoder 2 (105) always needs to have an HD size decoding capability, and is processed together with the video decoder 1 (104) as described above. The amount can be very large.

  Therefore, in this embodiment, the effective area of the second video information is limited as shown in FIG. As shown in FIG. 5 (a), the content creator encodes the video in such a way that only the necessary area is shown as the second video information, and the effective area as shown in FIG. 5 (b). Generate information. For example, by limiting the total value of the effective areas to 1/6 or less of the HD size, the effective value of the encoding becomes the SD size even if encoding is performed with the HD size.

  Specifically, for example, the case where the second video information is encoded in the MPEG2 video format will be briefly described. As shown in FIG. 6, in the MPEG2 video format, each frame of video is correlated in time and space, and a difference value is encoded. The unit in which the correlation is taken is a predetermined unit (macroblock) composed of luminance and color difference, and the invalid area may be encoded so as to be a skip macroblock (the code amount is minimum) within one frame.

  However, even if it is reproduced as the second video information as it is, the invalid area is not necessarily correctly synthesized by the method for determining the opacity in the previous period. Therefore, the valid area information shown in FIG. The multipliers 205 and 206 input the transparency α1 as a determining factor. In the multipliers 205 and 206, according to the effective area information, α1 = 0 is set in the non-effective area, that is, in the invalid area, so that the first video information is always displayed. In the effective area, the opacity determination method (chroma key or the like) is separately applied.

  As described above, according to the present embodiment, the second video information is encoded as an HD size so that the effective area is equal to or less than a predetermined ratio of the HD size, and the first video information and the second video information are also used by using the effective area information during reproduction. Since it is possible to synthesize video information, the processing amount of the decoder can be reduced even during execution of the picture-in-picture function, and desired image synthesis can be realized.

  In the present embodiment, in consideration of up to a slight variation in the coding result of the invalid area of the second video information, the valid area information is used regardless of the state, and the invalid area is unconditionally used. Although the case where the opacity is set to 0 has been described, for example, if the invalid area can be encoded in a uniform state, the second video information after decoding is limited by limiting the ratio of the effective area at the time of encoding. It is also possible to obtain the desired effect by applying the above-described opacity determination method (chroma key or the like) to all the regions. In the present embodiment, the case where the second video is encoded in the MPEG2 video format has been described. However, the format is not limited as long as the encoding can be performed by assigning a code to the effective area. Absent.

Next, a second embodiment of the present invention will be described.
Since the block diagram of the reproducing apparatus according to the present embodiment is the same as that shown in FIG. 1, detailed description thereof is omitted. The difference from the first embodiment is that the second video information has an SD size and is smaller than the size of the first video information on the screen. As described in the first embodiment, if the second video information is the SD size, the processing amount of the video decoder can be reduced as compared with the HD size. However, in the case of the SD size, the use area of the second video plane 202 is limited, and it is impossible to combine a plurality of sub-screens at different locations.

  Therefore, in this embodiment, as shown in FIG. 7A, the second video information is divided into four parts, for example, and different people are arranged in the respective areas, so that the video is encoded as the SD size. Also, as shown in FIG. 7 (b), position information and application timing are generated so that it is possible to know where the divided areas are finally arranged on the display of the first video information. To do. That is, the auxiliary information analysis unit 109 in FIG. 1 extracts the position information and application timing of the divided areas from the auxiliary information, and inputs them to the display video forming unit 108 via the control unit 111.

  FIG. 8 shows an example of the configuration of the display image forming unit 108 in this case. The same parts as those in FIG. 2 are denoted by the same reference numerals and detailed description thereof is omitted. The pasting position information of the divided area output from the control unit 111 is input to the position correction unit 801. The pixel-unit data read from the second video plane 202 is subjected to position correction in accordance with the position information in the position correction unit 801, and thereafter, opacity is applied and added.

  FIG. 9 shows a specific example in this case. The second video information is encoded in SD size, and the area is divided into four. The second video information remains SD size on the video plane 202, but the position correction unit 801 corrects the display position in units of areas, and finally displays a desired small area in an arbitrary place such as 901. In addition, it is possible to obtain a display video output as a picture-in-picture function.

  As described above, according to the present embodiment, it is possible to display child screens at a plurality of positions as the picture-in-picture function while limiting the processing amount by using the second video information as the SD size.

  In the present embodiment, the position correction unit 801 is configured to be disposed in the subsequent stage of the second video plane 202. However, the same effect can be obtained even if the position correction unit is disposed in the previous stage of the second video plane 202. it can. In this case, the decoded second video information is stored in the second video plane 202 after position correction.

Next, a third embodiment of the present invention will be described.
The present embodiment relates to a recording apparatus that records video information on a recording medium that is played back by the playback apparatus according to the first and second embodiments.

  FIG. 10 shows a block diagram of the recording apparatus of the present embodiment. Reference numeral 1001 denotes an analog input terminal, which is an analog input terminal to which the second video information is input from a camera or another device. Note that analog information is input here, but the present invention is not limited to this, and digital information may be input. Reference numeral 1002 denotes a digital input terminal to which the first video information broadcast or transmitted from the network is input. Reference numeral 1003 denotes A / D conversion means for converting analog information output from the analog input terminal 1001 into digital information. However, when the second video information is input, the A / D conversion means 1003 does not perform conversion. . Reference numeral 1011 denotes auxiliary information creation means for creating effective area information and application time information of the second video information. Reference numeral 1004 denotes an encoder that encodes digital second video information. Reference numeral 1005 denotes video information selection means for selecting encoded second video information or first video information output from the digital input terminal 1002. Reference numeral 1006 denotes medium selection means for selecting a recording medium. Reference numeral 1009 denotes an optical disk, and 1010 denotes a local storage such as a hard disk. Reference numeral 1007 denotes a user I / F that receives an instruction from the user. A control unit 1008 controls the entire recording apparatus. Note that the control unit 1008 may be shared with the control unit 111 of the playback unit 1012. Reference numeral 1012 denotes a playback unit that performs the playback-side operation described in the first and second embodiments, and includes a display video forming unit 108 and the like. The display image forming means 108 receives the second image information inputted from the analog input terminal 1001 and the first image information inputted from the digital input terminal 1002, and the effective area information created by the auxiliary information creating means 1011. The video to be displayed on the display screen is synthesized using the application time information.

  The present embodiment is characterized in that effective area information and application time information of the second video information are created and recorded together with the second video information on a recording medium. By doing in this way, the recording medium corresponding to Example 1 or Example 2 can be created. In addition, since the data capacity of the invalid area that is not the valid area is small, the burden on the control unit 1008 can be reduced. Furthermore, the burden on the control unit 111 can be reduced when video is reproduced from the recording medium as picture-in-picture.

  Note that the first video information is digital information output from the digital input terminal 1002. However, the first video information is not limited to this, and is input in analog as in the second video information and is converted by the A / D conversion means. It may be converted from analog to digital.

  In addition, the first video information and the second video information are recorded at the same time. However, after the first video information is recorded, the first video information is recorded from the recording medium in accordance with the reproduction of the first video information. Two pieces of video information may be input and recorded. Conversely, after the second video information is recorded, the first video information may be input from the recording medium and recorded in accordance with the reproduction of the second video information.

  Further, when receiving information in which the first video information and the second video information are multiplexed together with the auxiliary information, the range of the effective area of the second video information is increased or decreased, and the position of the range is changed. When changing, auxiliary information is separated, and effective area information and application time information included in the auxiliary information are changed and re-multiplexed and recorded. Further, when the second video information and the auxiliary information are in the format of the first embodiment, the format is changed to the format of the second embodiment, or conversely, the format of the first embodiment is changed to the format of the second embodiment. It is also one of the embodiments that the recording is changed to

  As described above, in the embodiment, the second video reproduced as a picture-in-picture is encoded as an HD size, and the effective area thereof is within a predetermined ratio with respect to the entire image. Thus, the moving picture stream of the second video is accompanied by auxiliary information such as area information of the effective area and application timing information thereof. The reproduction method and the reproduction apparatus include the auxiliary information analyzing unit, and in accordance with the extracted area information at the time of reproduction, the ineffective area other than the effective area in the second video is applied with transparency at an appropriate timing. Constitute.

  In addition, the second video of the second embodiment reproduced as a picture-in-picture is encoded as an SD size, and the moving picture stream of the second video is supplementary such as pasting position information of divided areas and application timing information thereof. It shall be accompanied by information. The reproduction method and the reproduction apparatus include the auxiliary information analysis unit, and when the second video is encoded as an SD size, the second video is appropriately timed according to the extracted position information. The predetermined divided area is displayed so as to be superimposed on the first video.

  When the second video to be played back as picture-in-picture is encoded as HD size, the effective area is substantially suppressed below a predetermined area of the entire image, and the encoding is performed while maintaining the image quality. The rate can be kept low. Since the ineffective area is rendered transparent at the time of reproduction, the display of the first video is not hidden outside the effective area as a picture-in-picture.

  Also, according to the playback method and apparatus, when the second video played back as a picture-in-picture is encoded as an SD size, the video stream of the second video is accompanied by the position information of the divided areas. A plurality of small area images can be displayed at desired positions as picture-in-pictures during reproduction. Even if the first video is HD size, since the second video is encoded as SD size, the encoding rate and the processing amount can be reduced.

  In the embodiments so far, the reproducing method, the reproducing device, the recording method, and the recording device have been described. However, a recording medium corresponding to any one of the first to third embodiments is also one of the embodiments. By using such a recording medium, it is possible to reduce the burden of reproduction processing.

  Moreover, although the example which uses both effective area information and application time information as auxiliary information was described as an Example, when applying a still image, it cannot be overemphasized that application time information does not need to be used.

  Further, the resolution size of the effective area of the second video information is set to be a predetermined ratio or less with respect to the resolution size of the first video information, but from the viewpoint of reducing the burden of the reproduction process, It goes without saying that the resolution size of the effective area of the second video information only needs to be smaller than the resolution size of the first video information. However, it is easier to design the playback apparatus to be a predetermined ratio or less.

  In addition, the first video information was transmitted from a broadcast or network, and the second video information was transmitted from another device such as a camera, broadcast, or network. However, the present invention is not limited to this. is not. For example, it may have been reproduced from the optical disc 1009 or the local storage 1010. Editing and recording on the same recording medium or recording on another recording medium is one embodiment.

  Further, the apparatus includes a receiving unit that receives the first video information and the second video information, and a display unit that displays the first video information and the second video information. A video display device corresponding to the playback device is also one embodiment.

The block diagram of the reproducing | regenerating apparatus of an Example. The figure which shows an example of a structure of the display image formation part by Example 1 of this invention. Basic image of the picture-in-picture function. The figure which shows the picture in picture function in case 2nd video information is HD size. An example of the effective area | region restriction | limiting of 2nd video information. The figure for demonstrating encoding of video data. An example of the division | segmentation of 2nd video information. The figure which shows an example of a structure of the display image formation part by the 2nd Example of this invention. The figure which shows the picture in picture function in case 2nd video information is SD size. FIG. 9 is a block diagram of a recording apparatus according to a third embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Optical disk 102 ... Local storage 103 ... PID filter 104 ... Video decoder 1
105 ... Video decoder 2
106 ... Graphics decoder 1
107 ... Graphics decoder 2
108 ... display image forming unit 109 ... auxiliary information analyzing unit 110 ... reference time counter unit 111 ... control unit 112 ... comparing unit 201 ... first video plane 202 ... second video plane 203 ... presentation graphics plane 204 ... interactive graphics plane 205, 206, 208, 209, 211, 212 ... multipliers 207, 210, 213 ... adders

Claims (12)

First decoding means for decoding encoded data of the first video information;
Second decoding means for decoding the encoded data of the second video information,
A playback device capable of outputting the second video information to a display device in synchronization with the first video information,
The screen size of the second video information is smaller than the screen size of the first video information,
The second video information is divided and displayed,
Display video forming means for forming a display video using the first video information and the second video information;
Control means for controlling the display screen forming means;
A playback apparatus comprising: The playback apparatus according to claim 1,
The size of the second video information on the screen is less than a predetermined ratio with respect to the size of the first video information on the screen. The playback apparatus according to claim 1,
Auxiliary information analyzing means for analyzing auxiliary information related to position information of a plurality of divided areas of the second video information,
The reproduction device according to claim 1, wherein the control unit controls the display of the second video information by giving an instruction to the display video forming unit using the auxiliary information. The playback apparatus according to claim 3, wherein
The auxiliary information has application time information for applying the auxiliary information,
This playback device
A reference time counter indicating a reference time for decoding and displaying video;
Comparing means for comparing the application time information and the reference time counter,
The control means controls the display position of each divided area of the second video information by giving an instruction to the display video forming means using the auxiliary information based on the output result of the comparison means. A playback device. The playback device according to any one of claims 1 to 4,
The display screen forming unit includes a first storage unit that stores the first video information, and a second storage unit that stores the second video information,
A reproduction apparatus characterized in that the outputs of the first storage means and the second storage means can be combined by switching the opacity in units of pixels. The playback device according to any one of claims 1 to 5,
The reproduction apparatus according to claim 1, wherein the first decoding means and the second decoding means are formed by a single decoding means. The playback apparatus according to any one of claims 1 to 6,
The reproduction apparatus according to claim 1, wherein the first decoding unit and the second decoding unit are realized by software. Decoding the encoded data of the first video information and the encoded data of the second video information;
A reproduction method capable of outputting the second video information to a display device in synchronization with the first video information,
The screen size of the second video information is smaller than the screen size of the first video information,
The second video information is divided and displayed,
A playback method, wherein a display video is formed using the first video information and the second video information. A recording method for recording first video information and second video information,
Creating auxiliary information regarding position information of a plurality of divided areas of the second video information;
Recording the created auxiliary information together with the second video information;
The screen size of the second video information is smaller than the screen size of the first video information,
A recording method comprising: recording the first video information and the second video information in synchronization on a display screen. The recording method according to claim 9, comprising:
The recording method according to claim 1, wherein a size of the second video information on the screen is equal to or less than a predetermined ratio with respect to a size of the first video information on the screen. A receiving unit for receiving the first video information and the second video information;
First decoding means for decoding encoded data of the first video information;
Second decoding means for decoding encoded data of the second video information;
A display unit that displays the second video information in synchronization with the first video information,
The screen size of the second video information is smaller than the screen size of the first video information,
The second video information is divided and displayed,
Display video forming means for forming a display video using the first video information and the second video information;
A video display device comprising: A recording medium on which first video information and second video information are recorded,
Auxiliary information which is position information of the second video information is recorded;
The screen size of the second video information is smaller than the screen size of the first video information,
The second video information is divided and displayed,
The recording medium, wherein the first video information and the second video information are displayed synchronously on a screen.

Images