JP3962053B2 - Video re-encoding device - Google Patents

Description

  The present invention relates to a moving image re-encoding device, and in particular, decodes moving image data obtained by encoding such as the MPEG2 system, and appropriately applies the obtained moving image signal directly or graphics overlay or special reproduction processing. The present invention relates to a moving image re-encoding device that performs re-encoding after the operation.

  In a playback device of a digital video system such as a DVD (digital versatile disc or digital video disc) system, compressed moving image data, audio data, sub-video data, etc. obtained from a playback signal are multiplexed into multiplexed data. On the other hand, each data is separated and decoded. The moving image signal (video signal) obtained by decoding the moving image data is subjected to D / A conversion and analog output after being subjected to an overlay process of a sub-picture or a GUI (graphical user interface) unique to the playback device. The Also, the audio signal obtained by decoding the audio data is D / A converted and output as an analog signal or is output as a digital signal.

  In recent years, digital interfaces in homes such as IEEE 1394 have been standardized, and development of digital TVs for digital broadcasting is also progressing. Due to the digitization of video equipment in the home and the interface between the equipment, video signal transmission between video equipment in the home, which has conventionally relied on analog signals, is becoming possible.

  On the other hand, there are various standards for digital video and digital audio encoding methods for each application, and the actual situation is that the standards are not sufficiently compatible. For example, the DVD recording format, the digital video camera format such as DVCR (Digital Video Cassette Recorder), and the digital broadcast format are not compatible. As for digital broadcasting, the US system (ATSC), the European system (DVB), and the Japanese system have different systems for each region, and compatibility of digital data is not guaranteed between these systems. Therefore, in order to connect various digital video equipment with a digital interface, it is necessary to perform conversion of a data format including conversion of an encoding method in accordance with each application.

  Data format conversion methods can be broadly classified into the following two types. The first conversion method is data format conversion processing at a digital data level that is encoded data. In this method, the degree of freedom of conversion may be greatly limited depending on the encoding method of encoded data before conversion. For example, performing digital processing such as special reproduction, effect processing, or video mixing on moving image data obtained by MPEG-2 encoding is extremely restrictive. Also, conversion at different digital data levels at the digital data level is often difficult to implement.

  The second data format conversion method is to decode the encoded data into a baseband signal, and encode the baseband signal again with a desired encoding method (this is called re-encoding). In this case, the degree of freedom of data format conversion becomes great, and special reproduction, effect processing, digital such as video mixing, etc. can be performed on the baseband signal, and there are no restrictions. However, since many encoding methods are irreversible conversions, the image quality and the sound quality are deteriorated each time the conversion is repeated in this method. In addition, in order to perform highly efficient re-encoding, it is necessary to install an additional encoding device in the digital video equipment. In particular, the addition of an encoding device with motion compensation such as an MPEG2 encoder is required for each digital device. Incurs significant cost increase for video equipment.

  As described above, in order to digitally connect digital video equipment having different data formats, when performing conversion of a data format including conversion of an encoding method, a method of performing conversion at a conventional encoded data (digital data) level However, there is a problem that the degree of freedom of processing such as special reproduction, effect processing, and video mixing is small, and the encoded data is decoded and special reproduction, effect processing, video mixing, etc. are performed at the baseband signal level. The method of re-encoding with a desired encoding method has a problem in that the image quality is deteriorated and the cost of digital video equipment is increased.

  The present invention has a degree of freedom for special reproduction and other processing by processing at the baseband signal level, suppresses image quality deterioration due to data format conversion including an encoding method, and further reduces the cost for additional re-encoding. An object of the present invention is to provide a moving image re-encoding device that can be significantly reduced.

  In order to solve the above-described problems, the present invention provides a special reproduction process such as fast-forward, slow, pause, sub-picture or GUI for a video signal which is a baseband signal obtained by decoding encoded video data. Low-cost, by determining the optimal encoding parameters based on various encoding parameter information extracted from the video encoded data at the time of decoding, information on special reproduction and overlay processing, etc. This makes it possible to realize high-quality re-encoding.

  That is, a first moving image re-encoding device according to the present invention includes decoding means for decoding a first encoded data obtained by motion compensation predictive encoding a moving image signal to obtain a moving image signal; Coding parameter extraction means for extracting a coding parameter including at least information about a motion vector, a coding mode and a quantization step from the first coded data; and a moving image signal obtained by the decoding means Re-encoding means for re-encoding in accordance with the encoding parameter extracted by the encoding parameter extraction means to obtain second encoded data, wherein the re-encoding means comprises (a) the extracted encoding Coding using the same coding parameter as the parameter, (b) Motion compensation prediction coding using a predetermined motion vector, and (c) Intra coding Characterized in that it has a coding mode selecting means for selecting over de each macroblock.

  In the first moving image re-encoding device, a motion vector detection circuit that requires a large amount of resources is not required for re-encoding, and a significant cost reduction can be realized.

  Also, in a region where no processing is applied to the video signal that is a moving image signal obtained by decoding, (a) is automatically automatically selected as the re-encoding encoding mode, and the first By matching the encoded data with all the encoding parameters, it is possible to minimize image quality degradation due to re-encoding. On the other hand, in a region where some processing has been performed on the video signal, (b) predictive coding using a predetermined motion vector or (c) intraframe coding is selected as a re-coding mode. There is a case. By setting a predetermined motion vector value according to the processing content for the video signal, it is possible to perform predictive coding with high coding efficiency without requiring a motion vector detection circuit.

  The second moving image re-encoding device according to the present invention generates a sub-video signal or a GUI (Graphical User Interface) image signal corresponding to the video frame or video field of the moving image signal obtained by the decoding means. And means for sequentially performing overlay processing of the sub-picture signal or the GUI image signal on the video frame or video field, and the re-encoding means is extracted by the encoding parameter extracting means. Storage means for temporarily storing the encoded parameters, and re-encoding the video frame or video field subjected to the overlay processing using the encoding parameters stored in the storage means, To do.

  In moving picture coding such as MPEG2, there are forward and backward predictions between frames, so that the frame order of encoding and decoding generally does not match the frame order of display. These differences in the frame order are absorbed by the frame rearrangement processing in each of the encoding device and the decoding device. In a system that performs graphics overlay and the like in display order, it becomes difficult to re-encode at the timing of decoding. In this case, since re-encoding is performed after decoding and graphic overlay processing, a predetermined delay occurs between certain decoding and re-encoding. However, by temporarily storing the extracted encoding parameter and delaying the extracted encoding parameter in the same manner, re-encoding using the same encoding parameter as the first encoded data becomes possible.

  A third video re-encoding device according to the present invention corresponds to the decoding order of video frames or video fields of a video signal obtained by the decoding means in the first video re-coding device. Means for generating a sub-video signal or GUI image signal, and means for sequentially performing overlay processing of the sub-video signal or GUI image signal on the video frame or video field, and the re-encoding means The method is characterized in that the video frame or video field subjected to the overlay processing is re-encoded according to the decoding order.

  As described above, because of the difference in the frame order of encoding and display such as MPEG2, delay due to frame rearrangement occurs in decoding and re-encoding, and the same as normal encoding for re-encoding Requires frame memory for frame rearrangement. On the other hand, in the third moving image re-encoding device according to the present invention, the sub-picture and GUI overlay processing is performed almost simultaneously with the decoding in the decoding order, and the re-encoding processing is further decoded and Since it can be performed simultaneously with overlay processing, it is possible to minimize the amount of delay associated with frame rearrangement, and the configuration does not require a memory for frame rearrangement by the re-encoding means. It becomes possible. The re-encoding means need only have a reference image frame memory for predictive encoding.

  The fourth moving image re-encoding device according to the present invention is the above-described first to third moving image re-encoding device, wherein the encoding parameter extracting means uses the first encoded data as the encoding parameter. Further, a frame / field adaptive orthogonal transform type is extracted, and the re-encoding means has a frame / field adaptive orthogonal transform means, and the encoding mode selection means (a) encodes the same encoding parameter as the encoding parameter. For a macroblock for which a coding mode using is selected, a frame / field adaptive orthogonal transform type extracted from the first coded data is selected, and (b) a predetermined motion vector is selected by the coding mode selection means. And a macroblock in which any coding mode of (c) intra-coding is selected. For by selecting an optimal frame / field adaptive orthogonal transform type, and characterized by applying orthogonal transformation respectively by the frame / field adaptive orthogonal transform means.

  As described above, in a region where no processing is applied to the video signal, it is possible to minimize image quality degradation due to re-encoding by matching the first encoded data with all the encoding parameters. It becomes possible. However, it is not always optimal to match the first encoded data with all the encoding parameters in a portion where the special reproduction or overlay processing is applied to the video signal. In such a portion, the frequency of automatically selecting the predictive coding based on the predetermined motion vector (b) or the intra coding (c) increases. Therefore, in the macroblock for which (b) or (c) is selected, the optimum type is re-selected for the field / frame adaptive orthogonal transform (DCT (discrete cosine transform) in MPEG2 etc.) and the selected type is used. By performing the encoding, it is possible to improve the encoding efficiency.

  The fifth moving image re-encoding device according to the present invention is characterized in that, in the first to fourth moving image re-encoding devices, the predetermined motion vector (b) is a zero vector.

  In special playback such as fast forward, slow, and top feed, the same frame of video may be displayed multiple times, and subtitles such as subtitles are displayed at the same position on the screen over several consecutive frames. Sub-pictures are often displayed on the screen. In any case, by using motion compensation using a zero vector, efficient predictive coding can be realized without a motion vector detection circuit.

  The sixth moving image re-encoding device according to the present invention is the first to fourth moving image re-encoding device, in particular, the predetermined motion vector (b) is the display position of the sub-picture signal or the GUI image. And the amount of movement thereof.

  In advanced sub-pictures and GUI images that are overlaid on video signals, it may be possible to move a still picture in time within a video frame. In the sixth moving image re-encoding device, it is possible to calculate an optimal motion vector from the amount of motion of the overlaid image according to the macroblock position of the overlaid region, and the default motion vector of (b) It is possible to improve the encoding efficiency by setting the optimal motion vector value to the value of.

  In a seventh moving image re-encoding device according to the present invention, in the first to fourth moving image re-encoding devices, the encoding mode selecting means displays the display position of the sub-picture signal or the GUI image signal and The encoding mode is selected by using at least one of a motion amount, an area ratio of each moving image signal obtained by the decoding means to each macroblock, and a mixing ratio.

  In the first to fourth moving image re-encoding devices, (a) encoding using the same encoding parameter as the encoding parameter, (b) motion compensated predictive encoding using a predetermined motion vector, ( c) In order to select one of the intra codings for each macroblock, for example, the motion compensation prediction error power in the encoding of (a), the motion compensation prediction error power in the encoding of (b), and the code of (c) It is necessary to calculate the signal power and the like of the encoding target macroblock in the encoding, and to determine the optimum mode by comparing them.

  Here, in the sixth moving image re-encoding apparatus, the comparison between the encoding modes (a) and (b) is performed using the overlay ratio instead of the error power, and the selected encoding mode and (c) The intra coding can be compared using prediction error power or the like. The overlay ratio is determined based on the area ratio of the overlay pixels in the macroblock and the mixing ratio interest rate when mixing with the video signal is performed. If the overlay ratio is equal to or higher than the threshold, the mode (b) can be selected, and if not, the mode (a) can be selected. As a result, the number of errors and signal power calculations can be reduced from 3 to 2 for each macroblock.

  An eighth moving image re-encoding device according to the present invention is the first to fourth moving image re-encoding devices, wherein the re-encoding unit obtains at least a part of the moving image signal obtained by the decoding unit. Resolution conversion means for performing resolution conversion for enlargement or reduction, and scaling for the motion vector extracted from the first encoded data or the predetermined motion vector according to the conversion ratio of the resolution conversion means A scaling unit, and the encoding mode selection unit selects the encoding mode using a motion vector after the scaling is performed.

  When the resolution conversion is performed after decoding and the video signal is re-encoded and output, the motion vector and the encoding parameter of the first encoded data cannot be used as they are. However, by appropriately scaling the motion vector information according to the resolution conversion ratio, it is possible to obtain a substantially appropriate motion vector without performing motion vector detection at the time of re-encoding.

  According to a ninth moving image re-encoding device of the present invention, in the first to fourth moving image re-encoding devices, at least a part of the moving image signal obtained by the re-encoding unit by the decoding unit. A rate conversion means for converting a display frame rate or a display field rate with respect to a motion vector extracted from the first encoded data or a scaling with respect to the predetermined motion vector according to a conversion ratio of the rate conversion And the encoding mode selection unit selects the encoding mode using a motion vector after the scaling is performed.

  When the decoded video signal is re-encoded and output after frame frequency or field frequency conversion is performed by frame or field thinning, repeated display, interpolation, or the like, the first encoded data is used. If the extracted motion vector and encoding parameter are used as they are, image quality deterioration may be caused. Corresponding to these frame or field frequency conversions, the extracted motion vector is subjected to conversion processing including scaling, whereby the image quality deterioration can be suppressed.

  According to a tenth moving image re-encoding device of the present invention, in the first to fourth moving image re-encoding devices, the re-encoding unit has fast-forwarding, slowing, top-feeding, and pausing that the decoding unit has. Motion vector conversion means for converting a motion vector extracted from the first encoded data in accordance with a playback parameter related to a special playback function including at least one of the encoding mode selection and the encoding mode selection The means selects the coding mode using the motion vector converted by the motion vector conversion means.

  During special playback, processing such as deleting one field of the interlaced image and displaying the remaining fields continuously, thinning the frame or field to be decoded, or displaying the decoded frame or field multiple times, etc. Done. In the tenth moving image re-encoding device, it is possible to suppress deterioration in image quality when re-encoding a special reproduction image by performing motion vector conversion according to these special reproduction processes.

  In an eleventh moving image re-encoding device according to the present invention, in the first to fourth moving image re-encoding devices, the re-encoding means encodes a frame or macroblock unit code from the first encoded data. Data extraction means for extracting encoded data; and header data changing means for changing predetermined header data including at least data related to the display order and the configuration of the display field for the encoded data extracted by the data extraction means; Data replacement means for extracting a part of the second encoded data by the data extraction means and replacing the header data with the encoded data in which the header data has been changed by the header data change means.

  For each of the encoded image frame and the reference image frame related to the encoded frame, the same encoding parameter as that of the first encoded data is used for a frame or a region that is re-encoded without any processing after decoding. By re-encoding using, the degradation of image quality due to re-encoding is sufficiently small. However, it may not be completely reversible due to the influence of rounding, limiter processing, etc. due to the calculation accuracy in the decoding process or the re-encoding process.

  In the eleventh moving image re-encoding device, in the re-encoded second encoded data, the region in which no processing is performed on the video signal as described above, the corresponding first encoding By substituting with data and outputting, it is possible to completely remove even slight image quality degradation caused by calculation accuracy. In addition, when processing such as special reproduction processing or frame rate conversion is performed, in order to satisfy the validity of the output encoded data, the header area of the first encoded data is included as necessary. It is possible to output the first encoded data as re-encoded data by appropriately changing the value of the predetermined data indicating the display order and the display field configuration.

  A twelfth moving image re-encoding device according to the present invention is the eleventh moving image re-encoding device, wherein the encoded data replacement means extracts the motion vector extracted from the first encoded data, The coded data is replaced in units of macroblocks from the display position of the video signal or the GUI image signal and the amount of movement thereof.

  In the twelfth moving image re-encoding apparatus, it is possible to specify both the region to be subjected to overlay processing and the region to which each macroblock of the decoded moving image refers in the motion compensated prediction encoding. From these pieces of information, it becomes possible to discriminate macroblocks that are affected by overlay processing and macroblocks that are not affected, and output the first encoded data as it is or re-encoded second encoding Whether to output data can be switched for each macroblock and re-encoded.

  In a thirteenth moving image re-encoding device according to the present invention, in the eleventh moving image re-encoding device, the encoded data replacement means is at least one of fast-forwarding, slowing, top-feeding, and pausing of interlaced moving image signals. The second encoded data is selected and output at the time of special reproduction including one, and the header data is extracted by the data extraction means and the header data is changed by the header data changing means at the time of normal reproduction and special reproduction of the progressive video signal. The changed encoded data is selected and output.

  Here, the interlaced video signal is a video at a time when the video of the even-numbered line and the odd-numbered line in the frame are separated by a half time of the frame period, and the progressive video signal is the even-numbered video in the frame. The video of the line and the odd line is the video at the same time.

  In slow playback and top feed, processing for repeatedly displaying the same video frame is performed. Also, in fast-forward playback, the same video frame repeated display as frame thinning may be performed in combination depending on the transfer rate of encoded data. In this way, when the same video frame is repeatedly displayed, when a moving image is displayed in an interlaced frame, the display moves jerky because there is a motion in the even and odd lines. It will be. Usually, processing for smoothly displaying is performed by selecting and repeatedly displaying either even lines or odd lines. However, with motion-compensated predictive encoded data of interlaced video signals, it is difficult to extract only one line (ie, field) video at the encoded data level.

  In such a case, smooth display can be realized by outputting the second encoded data obtained by encoding the baseband moving image signal after the special reproduction processing according to the present invention. In addition, since there is no video motion within a frame in a progressive image, special reproduction can be realized by rearranging the first encoded data and changing / modifying header data. By adaptively combining the two and outputting, it is possible to minimize image quality degradation accompanying re-encoding and to prevent image disturbance during special reproduction.

  A fourteenth moving image re-encoding device according to the present invention is the first to fourth moving image re-encoding devices, wherein the re-encoding means has an inter-frame prediction structure different from that of the first encoded data. Means for performing re-encoding, and means for performing scaling on a motion vector extracted from the first encoded data in accordance with an inter-frame prediction structure at the time of re-encoding, wherein the encoding mode The selection unit selects the encoding mode using the scaled motion vector.

  In this way, in order to reduce the cost and delay time of the re-encoding unit, even if the inter-frame prediction structure at the time of re-encoding is different from that of the first encoded data, it is extracted from the first encoded data. Can be re-encoded by predictive coding using motion compensated prediction without re-detecting a new motion vector at the time of re-encoding. It becomes.

  A fifth moving image re-encoding device according to the present invention is the first to fourth moving image re-encoding devices, wherein the decoded video corresponding to the encoded frame in the first encoded data is used. According to another aspect of the invention, there is provided means for switching display of the sub-video signal or the GUI image signal subjected to the overlay processing in accordance with a boundary of a frame or a video field.

  When the video signal after overlay processing is re-encoded with the same frame configuration as the first encoded data, when the encoded data obtained by re-encoding is decoded, the displayed sub-video or GUI image is In some cases, the image may be disturbed with an unnatural time change.

  The fifteenth moving image re-encoding device shifts the change point of the sub-video signal or the GUI image to the boundary of the field constituting the encoded frame, thereby converting the overlay-processed video signal into the first encoded data. Even when re-encoding with the same frame configuration, sub-video signals and GUI images are not disturbed in time when a video signal obtained by decoding the re-encoded data is displayed, and a smooth video is displayed. It becomes possible.

  According to the present invention, a coding parameter including a motion vector, a coding mode, and a quantization step is extracted at the time of decoding the encoded moving image data, and a sub-picture or GUI is extracted from the decoded baseband video signal. For frames or regions that have been subjected to processing such as overlay and have not been processed, either re-encode using the extracted encoding parameters, or modify part of the original encoded data and output However, for regions that have undergone overlay processing, etc., re-encoding is performed by converting and generating motion vectors according to the contents of each processing, so that the degree of freedom of overlay processing is high, and high-quality and low-cost re-encoding is performed. Encoding can be realized.

  Therefore, the moving image re-encoding device of the present invention makes it possible to smoothly perform digital connection between digital video systems of different systems.

Embodiments of the present invention will be described below with reference to the drawings.
[First Embodiment]
FIG. 1 shows the configuration of a system including a moving image re-encoding device according to the first embodiment of the present invention. In this system, a digital video / audio reproduction device such as a conventional DVD player has a digital output interface to the IEEE1394 bus in addition to analog video and analog audio output. Here, a portion surrounded by a broken line 28 in the figure is a re-encoding unit added to the conventional digital video / audio reproduction device.

  The optical disk 50 is, for example, a DVD disk, and audio data, encoded moving image data, sub-picture data, navigation data, and the like are multiplexed and recorded. A signal reproduced from the optical disc 50 is input to the data separator 10, and the audio data 51, moving image data 52, sub-video data 53, and navigation data 54 are separated. The separated data 51, 52, 53, and 54 are decoded by the audio decoder 15, the video decoder 16, and the sub-picture decoder 17 through the buffer memories 11, 12, 13, and 14, respectively.

  With respect to the moving picture signal 56 obtained by decoding the moving picture data by the video decoder 16, a sub-picture signal 57 obtained by decoding the sub-picture data by the sub-picture decoder 17 and a reproduction apparatus by a GUI (Graphical User Interface) The GUI image signal 59 uniquely generated by the image generator 20 in response to the user's operation 60 is subjected to overlay (synthesis) processing by the video overlay processing unit (OSD) 18 via the adder 19.

  FIG. 3 shows an example of an overlay image for the moving image signal 56 obtained by decoding. 3A shows a frame (moving image frame) of the moving image signal 56, and FIG. 3B shows a moving image frame after overlay. A subtitle portion 111 in FIG. 3B is a sub-picture signal encoded as a bitmap picture, decoded by the sub-picture decoder 17 and called a sub-picture in the DVD system. Also, the time display 110 in FIG. 3B is not recorded on the optical disc 50, but is a graphic overlay of the time generated by the playback device itself on the video. Returning to FIG. 1, the audio signal 55 obtained by decoding the audio data by the audio decoder 15 and the moving image signal 58 overlay-processed by the video overlay processing unit 18 are respectively converted into the D / A converter 29, 30 is converted into analog signals 61 and 62 and output to the outside. Further, the audio signal 55 and the moving image signal 58 after the overlay processing are re-encoded by the audio encoder 21 and the video encoder 22, respectively. Audio data and moving image data obtained by this re-encoding are multiplexed into a transport stream by the multiplexing unit 26 via the buffer memories 24 and 25, respectively, and further digitally output from the IEEE 1394 interface 27.

  The video decoder 16 extracts a coding parameter 64 included in the moving image data 52 before decoding such as a motion vector for each macroblock, a coding mode, and a quantization step simultaneously with the decoding. The parameter 64 is temporarily stored in the data memory 31. The video encoder 22 reads out encoding parameters corresponding to image frames and macroblocks to be re-encoded from the encoding parameter data temporarily stored in the data memory 31, and provides predetermined motion vector information given from the outside. 65 is input together.

(First Configuration Example of Video Encoder 22)
FIG. 2 is a diagram showing a first configuration example of the video encoder 22 for re-encoding in FIG. In the figure, components given the same reference numerals as those in FIG. 1 indicate the same components as those in FIG. This video encoder 22 is based on a motion compensated predictive orthogonal transform coding system represented by MPEG2 or the like, and includes a motion compensation unit 70, a motion compensation mode (coding mode) and a DCT type determination unit 72, a DCT and a quantization unit. 74, an inverse quantization and inverse DCT unit 75, a frame memory 23, a rate control unit 73, and a variable length coding unit 76.

  The video encoder 22 is characterized in that it does not have a motion vector detection circuit necessary for a moving picture coding apparatus such as a normal MPEG2 encoder. In a normal moving image encoding apparatus, a huge block matching operation is required to detect a motion vector for each macroblock. In general, the hardware resources that the motion vector detection circuit occupies in the entire coding apparatus occupy a very high ratio, but the video encoder 22 does not require such a large hardware motion vector detection unit. That is, in this embodiment, motion compensation is performed using both the motion vector 64 extracted from the moving image data 52 before decoding and the predetermined motion vector 65 given from the outside.

  Next, the motion compensation mode and DCT type determination unit 72 uses which of the motion vectors 64 and 65 is used based on the power or magnitude of the prediction error signal by each motion compensation and the power or magnitude of the encoding target image signal. Alternatively, whether to perform intra coding is determined for each macroblock.

  Next, for a macroblock for which predictive encoding using the motion vector 64 extracted from the video data 52 before decoding is selected, the DCT type described later matches the DCT type extracted from the video data 52 before decoding. In other cases, optimal DCT type detection is performed.

  The rate control unit 73 determines a quantization step for each macroblock used in the quantization unit 74. That is, for a macroblock for which predictive encoding using the motion vector 64 extracted from the moving image data 52 before decoding is selected, the quantization extracted from the moving image data 52 is not subject to the output bit rate restriction. When the steps are used as they are and the output bit rate is restricted, the quantization step is determined with reference to the generated code amount 80 so that the desired bit rate is obtained. For a macroblock for which a predetermined external motion vector 65 is selected or for which intra coding is selected, a quantization step extracted from the moving image data 52 when the output bit rate is not limited, Alternatively, when a predetermined predetermined quantization step is used and the output bit rate is restricted, the quantization step is determined with reference to the generated code amount 80 so as to obtain a desired bit rate.

(Operation of video re-encoding device)
FIG. 4 shows an example of operation timing of the video re-encoding device according to the present embodiment. Here, both decoding and re-encoding are based on MPEG2. In the figure, I represents an intra-coded picture (I picture), P represents a forward predictive coded picture (P picture), B represents a bi-directional predictive coded picture (B picture), and the subscript number represents the number of each frame. The order of display is shown.

  In FIG. 4, reference numeral 101 (Video DTS) indicates the timing at which each frame is decoded, and the time is described in the encoded data as DTS (Decoding Time Stamp: decoding time management information). In MPEG2, the order of encoding and decoding is generally different from the order of display for inter-frame prediction of B pictures. In the example of FIG. 4, reference numeral 101 (Video DTS) is the timing at which the main video of each frame is displayed, and the time is described in the encoded data as PTS (Presentation Time Stamp: reproduction output time management information). Has been. In MPEG2, the PTS of the B picture matches the DTS of the same frame, and the PTS of the I picture and the P picture matches the DTS of the I picture or P picture to be decoded next.

  In the DVD, the sub video signal mapped to the main video signal is decoded in the display order, and the time DTS coincides with the sub video display time PTS. Reference numeral 102 (SP DTS) in FIG. 4 indicates the decoding and display timing of the sub-video signal corresponding to each moving image frame. The sub video signal is overlaid on the main video signal simultaneously with decoding. Reference numeral 103 (OSD) in FIG. 4 indicates the timing of overlay processing.

  In the example of FIG. 4, the sub video signal corresponding to each moving image frame is decoded and displayed as a video signal after overlay, but the sub video frame may not exist for all moving image frames. Also, it is possible to repeatedly display the same sub-video signal for a certain period of time, or to highlight a part of the sub-video signal according to the navigation process.

  The output of the analog video signal can be started at the same time as the overlay processing. Therefore, reference numeral 104 (Display (Analog)) in FIG. 4 indicates the timing at which the analog video signal is displayed.

  On the other hand, in order to re-encode the video signal after overlay processing with the same parameters as the first encoding including each picture type of I, P, and B, the same order as the decoding order indicated by reference numeral 100 Need to be re-encoded. Therefore, re-encoding can be started from the time when the frame I2 to be decoded first is displayed. Reference numeral 105 (Video Re-encoding) in FIG. 4 indicates the re-encoding timing of the video signal after overlay processing. The re-encoded moving image data is multiplexed together with the re-encoded audio signal and output as digital data.

  On the receiving side (not shown), the encoded digital data is decoded after transmission / reception buffer delay and transmission path delay, and further displayed after frame rearrangement accompanying decoding is performed. Reference numeral 106 (Video Re-decoding) in FIG. 4 indicates the decoding timing on the receiving side, and reference numeral 107 (Display (Digital)) is displayed.

[Second Embodiment]
FIG. 5 is a block diagram showing a configuration of a system including a moving image re-encoding device according to the second embodiment of the present invention. In FIG. 5, the same components as those in the first embodiment shown in FIG. 1 are given the same reference numerals, and differences from the first embodiment will be described.

  The difference between the present embodiment and the first embodiment is that a selector 67 for selecting either the output video signal 58 of the video overlay processing unit 18 or the output video signal 66 from the frame memory 23 is added. The data memory 31 for temporary storage of the conversion parameters is removed, and the buffer memories 32 and 33 for the sub-picture data 53 and the navigation data 54 are changed from FIFO (First In First Out) memory to RAM (Random Access Memory). And the capacity of the buffer memories 11 and 12 for the audio data 51 and the moving image data 51 is increased.

  Further, the difference in operation is that the processing of the overlay processing unit 18 is performed not in the order of frames displayed as in the first embodiment but in the order of frames in which the moving image data 52 is decoded. .

(Operation of video re-encoding device)
Hereinafter, the operation of the present embodiment will be described with reference to FIG.

  FIG. 6 is a diagram showing the operation timing of the present embodiment, and corresponds to FIG. 4 showing the operation timing of the first embodiment. 6, reference numerals 120, 121, and 122 denote moving image data decoding timing, moving image data display timing, sub-picture decoding and display corresponding to the reference numerals 100, 101, and 102 in FIG. Each timing is shown.

  In the first embodiment, decoding and display are performed according to the timings 100, 101, and 102 shown in FIG. 4, whereas in this embodiment, decoding and re-encoding are performed at the timings indicated by reference numerals 123 to 126. The analog video image signal and digital video data are output.

  First, the decoding of each moving image data is not delayed from the original DTS, but is delayed until the PTS timing of the moving image frame I2 to be decoded first. Depending on the delay time of the decoding start time, the moving image data 52 read from the optical disc 50 is accumulated using the capacity added to the buffer memory 12. The same applies to the audio data 51, the sub-video data 53, and the navigation data 54. The reference numeral 123 in FIG. 6 is the moving image data decoding timing in the present embodiment.

  Next, the decoding and overlay processing of the sub-picture signal that is originally subjected to the overlay processing in the order of the displayed frames is performed according to the order in which the moving image frames are decoded. Reference numeral 124 in FIG. 6 indicates the decoding timing of the sub-picture signal and the overlay processing timing for the decoded moving image frame. Since the sub-picture decoding order is different from the original decoding order, the sub-picture data 53 and the navigation data 54 are read from the buffer memories 32 and 33 by randomly accessing necessary data.

  The analog video output 68 to be sent to the analog signal system 62 is switched by the selector 67 between the video signal 58 subjected to overlay processing and the video signal 66 input to the encoder 22 and recorded in the frame memory 23. The selector 67 selects the video signal 58 when the moving picture frame to be output is a B picture, and selects the video signal 66 read from the frame memory 23 when the moving picture frame is an I picture or P picture. As a result, the moving picture signals 58 output in the decoding order can be rearranged in the display order and output in analog form. Reference numeral 125 in FIG. 6 indicates the output timing of the analog video signal.

  On the other hand, the video signal 58 output from the overlay processing unit 18 in the decoding order is input to the re-encoding unit 34, and re-encoding is performed in the input order, that is, the decoding order. The encoder for re-encoding 22 has the same basic configuration as that of the first embodiment, and uses an encoding parameter 64 extracted from the first encoded data and a predetermined motion vector 65 given from the outside. Each macroblock is adaptively encoded. In the present embodiment, since re-encoding is performed simultaneously with decoding, the data memory 31 for temporarily storing the encoding parameters in FIG. 1 is not necessary. Further, since frame rearrangement for re-encoding is not required, the frame memory for frame rearrangement required in the first embodiment is not required, and the delay amount from the start of decoding is set to the first amount. It becomes possible to make it smaller than the embodiment. Reference numerals 127 and 128 in FIG. 6 indicate timings when data re-encoded on the receiving side is input, decoded, and displayed.

(For frame / field adaptive DTC and DCT types)
FIG. 7 is a diagram for explaining a DCT (discrete cosine transform) type used in MPEG2 or the like as a representative example of orthogonal transform. In MPEG2, a macroblock composed of 16 pixels × 16 lines in a frame is divided into four luminance blocks composed of 8 pixels × 8 lines and two color difference blocks, and each is subjected to DCT processing. Here, whether the luminance block is a block composed of frame lines or a block composed of lines of each field can be changed for each macroblock.

  In this way, changing a luminance block to be a block composed of frame lines or a block composed of lines of each field for each macroblock is called a frame / field adaptive DCT, and which block is Information indicating whether to make a luminance block is called a DCT type. FIG. 7A shows a macro block having a frame configuration, FIG. 7B shows division into frame blocks, and FIG. 7C shows division into field blocks.

  In the first and second embodiments of the present invention described above, the macroblock re-encoded using the motion vector extracted from the first encoded data is also the first DCT type. The DCT type extracted from the encoded data is used to adaptively select which DCT type to use for a predetermined motion vector or a macroblock for which intra coding is selected. The determination of these DCT types is performed by the motion compensation mode and DCT type determination unit 72 in FIG. In the DCT type adaptive selection, the correlation between lines in adjacent frames and the correlation between adjacent lines in the same field are calculated. If the intra-frame line correlation is large, the DCT of the frame block and the intra-field line correlation are calculated. Is larger, each field block DCT is selected.

(How to give default motion vector)
Next, a method for specifying the default motion vector 65 in FIGS. 1, 2 and 5 will be described.
In the first designation method of the default motion vector, a vector having a size of zero is given as the default motion vector. In general, sub-pictures such as subtitles and GUI images that are subjected to overlay processing on a decoded video signal are generally displayed stationary for a certain period. In such an area subjected to overlay processing, encoding efficiency is improved by performing predictive encoding with a motion vector having a motion amount of zero, rather than a motion vector extracted from the first encoded data. . In addition, in a frame where display or non-display of an overlay image is switched, prediction efficiency between frames generally decreases. In such a portion, intra coding may be performed instead of prediction coding.

  Therefore, by selecting the optimum one for each macroblock from among motion compensated predictive coding using the motion vector extracted from the first coded data, predictive coding using zero vector, and intra coding, overlay processing is performed. Even when the above is performed, the optimum encoding mode is automatically selected.

  In the second method for specifying a predetermined motion vector, the predetermined motion vector is determined from the display position and the amount of motion of the image to be overlaid. The former indicates at which position in the screen the sub-picture or the GUI image is overlaid, and the latter indicates the amount of movement when the overlaid image moves in time.

  FIG. 8 shows an example of the time change of overlay processing. In FIG. 8, it changes in order of (a)-> (b)-> (c) with progress of time. In FIG. 8B, overlay display is started, and time 110 and subtitle 111 are displayed. The subtitle 111 scrolls in the horizontal direction and moves to the left as shown in the subtitles 113 and 114 in the frame of FIG. The display positions at times 110 and 112 are fixed, and in the encoding of the macroblock corresponding to these positions, a predetermined motion vector 65 is set as a zero vector. For the subtitles 111, 113, and 114 to be scrolled, the scroll amount is given as a predetermined motion vector. By doing in this way, it is possible to improve the encoding efficiency by improving the prediction efficiency at the time of re-encoding even for an overlay image that involves temporal parallel translation.

(Second motion compensation mode determination method)
Next, the second motion compensation mode determination method in the motion compensation mode and DCT type determination unit 72 of FIG. 2 will be described.
The motion mode is determined by selecting a motion vector with higher prediction efficiency from the motion vector 64 extracted from the first encoded data and the predetermined motion vector 65. In the above description, a method using a vector having a smaller prediction error is used. In this case, however, the encoding target macroblock image is read, the reference macroblock image is read according to each motion vector, and each prediction error is determined. It is necessary to perform the operation.

  On the other hand, in the second motion compensation mode determination method, which motion vector is used is selected for each macroblock from the area ratio of the overlay image in the macroblock and the mixing ratio with the video signal. Thereby, it is possible to reduce the amount of calculation for determining the encoding mode.

  FIG. 9 is a diagram illustrating an example of encoding mode determination. In FIG. 9, reference numeral 111 denotes a portion to be subjected to overlay processing, and reference numerals 130 and 131 denote enlarged macroblocks at the time of re-encoding. Since most of the pixels in the macro block 131 are subjected to overlay processing, the original video signal is almost overwritten. In such a macroblock, it is possible to improve the prediction efficiency by using a predetermined motion vector that matches the overlaid image, rather than a motion vector extracted from the first encoded data.

  On the other hand, as shown in the macro block 130, when the ratio of the overlay area in the macro block is low, it is possible to maintain the encoding efficiency by using the motion vector extracted from the first encoded data. It becomes. A motion vector extracted from the first encoded data is also used for a macroblock that is not subjected to overlay processing at all.

  Furthermore, when the overlay processing is not pixel overwriting but mixing with the decoded video signal, the motion vector is selected using the product of the area to be overlaid in the macroblock and the mixing ratio. To decide.

(About the video encoder 22 having a motion vector scaling function)
FIG. 10 is a diagram illustrating a second configuration example of the video encoder 22 in FIGS. 1 and 5. 10, the same reference numerals as those in FIG. 2 denote the same components as those in FIG.

  The difference from the first configuration example of the video encoder 22 shown in FIG. 2 is that a motion vector scaling unit 77 for the motion vectors 64 and 65 is added. The video encoder shown in FIG. 10 uses a case where all or part of a video signal is output after resolution conversion, that is, enlargement or reduction, for decoding of moving image data and overlay processing by a decoding unit, or a frame frequency (frame rate). ) Or when the field frequency (field rate) is converted and output.

  That is, when re-encoding a moving image signal that has been subjected to enlargement or reduction resolution conversion, the motion vectors 64 and 65 correspond to the video resolution conversion ratio (enlargement or reduction ratio) by the motion vector scaling unit 77. Used after scaling.

  FIG. 11 shows an example in which an enlarged display image is output in which the lower left portion of the decoded and overlaid video frame 140 is doubled both horizontally and vertically. The macroblock 142 in the video frame 140 is re-encoded as four macroblocks 143 in the enlarged video frame. In this case, the motion vector value corresponding to the macroblock 142 is scaled twice in both the horizontal and vertical directions, and the scaled motion vector is adapted to each of the four macroblocks 143 and re-encoded.

  In addition, when re-encoding a moving image signal that has undergone display frame rate or display field rate conversion, input motion vectors 64 and 65 are also converted by the motion vector scaling unit 77 into display frame rate or display field rate. It is used after being scaled according to the ratio.

  That is, in the case where the decoded video signal shown in FIG. 12A is thinned out frame by frame and output as shown in FIG. 12B, the motion vector detected from the first encoded data is used. The video signal shown in FIG. 22B is re-encoded using the motion vector 202 obtained by scaling 201 out of 200 and 201 both horizontally and vertically.

(About special playback)
Next, in the first and second embodiments according to the present invention, the operation in the case of having special playback functions such as fast forward, reverse playback, pause, slow playback and the like will be described.

  FIG. 13 shows an example of fast-forward playback, in which only the I picture portion is extracted from the encoded moving image data and is decoded intermittently. Output the video of the specified frame. In FIG. 13, I pictures are encoded at 15-frame intervals, and decoding is performed at a cycle of 5 frames to realize triple-speed fast-forward playback. In the output frame indicated by hatching in FIG. 13, the video of the I picture decoded immediately before is output repeatedly. In this case, when re-encoding, the hatched frame can be re-encoded as a P picture instead of an I picture. Since it is originally an I picture, the first encoded data generally does not have motion vector information. However, since the prediction error becomes zero by motion compensation using the zero vector, the motion compensation mode of the zero vector is automatically selected, and the encoding efficiency at the time of re-encoding becomes very high. It becomes possible to reduce the transmission bit rate after re-encoding.

  FIG. 14 shows an example of reverse playback at 3 × speed, and reverse playback is performed by decoding only the I picture and reversing the order of frames to be decoded at that time in the same manner as fast forward in FIG. .

  FIG. 15 shows an example of a pause operation. In the example of FIG. 15, a pause state is obtained by repeatedly displaying the same frame after decoding the frame I2. Also in this case, at the time of re-encoding, the hatched frame is re-encoded as a P picture in the same way as in FIG. 13, and predictive encoding with a zero vector is automatically selected.

  FIG. 16 shows another method for realizing fast-forward playback. In FIG. 16, encoded data of only an I picture and a P picture is decoded and output, and a frame indicated by hatching in the figure repeatedly outputs an image of the immediately preceding frame. Therefore, in the example of FIG. 16, it is possible to realize gentle fast-forward playback at 1.5 times speed. Here, the hatched frame is re-encoded as a P picture using a zero vector, as in FIG. In addition, a frame that is not shaded is re-encoded without reducing the encoding efficiency by re-encoding using the motion vector and encoding parameter extracted from the first encoded data. It is possible.

  FIG. 17 shows an implementation example of slow playback. In the example of FIG. 17, decoding of all encoded moving image data is performed at a timing that is an integral multiple of the frame period, and the same display frame is repeatedly displayed when decoding is not performed. The hatched frame in FIG. 17 is a frame that is repeatedly displayed. Here, decoding is performed every three frames to realize slow reproduction at 1/3 speed. The signal indicated by Output1 in FIG. 17 is an analog output video signal. On the other hand, the re-encoded output at the time of slow reproduction has a frame configuration indicated by Output 2 in FIG. 17 in consideration of the inter-frame prediction structure.

  Here, the B0 and B1 frames are re-encoded as the same moving image frame as a B picture, and P2 * is the same moving image frame as I2 as a P picture. Since the reference image matches the predicted image, by performing motion compensation predictive coding of the zero vector, the prediction error signal is always zero and coding with very high coding efficiency is performed. The moving image data re-encoded as Output 2 is subjected to frame rearrangement according to a normal decoding procedure on the receiving side, and is reproduced as a 1/3 speed video similar to Output 1.

  At the time of re-encoding in FIGS. 13 to 17, motion-compensated predictive encoding using a motion vector extracted from the first encoded data or a zero vector is performed, and each encoded frame is determined according to the number of frame repetitions. The header data indicating the display order (referred to as temporal_reference in MPEG2) is rewritten.

  Also, in MPEG2, there is a rate conversion unit that performs conversion from an encoded frame rate to a display frame rate called 3: 2 pulldown, and whether each encoded frame is displayed for 3 field periods or 2 field periods. And a flag indicating whether each frame is displayed from the top field or the bottom field (top_field_first in MPEG2). In displaying the decoded video signal, these flags need to be set correctly so that the top field and the bottom field are always displayed alternately. In the examples of FIGS. 13 to 17, since the decoded frame is repeatedly displayed, the above header data flag is also repeatedly displayed so that the top field and the bottom field are always displayed alternately. Rewrite with appropriate modification according to the display structure.

  In the special reproduction described above, when the encoded moving image data is an interlaced video, if the decoded video signal is output as it is as described above, the repetitive frame shown in FIGS. In playback, there may be a jerky image of movement. This is because, in an interlaced frame, the second field of the same frame is an image that has passed one field time from the first field, and when the first field is displayed again, the displayed image is displayed in the past. This is because the time will go back and forth. Usually, in order to prevent such an unnatural movement, one of the first field and the second field is selected in the same frame, and the same field is used in the first field and the second field. A method of outputting an image is taken. In this case, each macroblock of the first encoded data does not always match each macroblock at the time of re-encoding.

  As described above, when a frame is formed by repeating the same field during special reproduction of an interlaced image, re-encoding is performed using the method shown in FIG. In FIG. 18, 0, 1, 2, and 3 indicate field numbers, and (a), (b), (c), and (d) indicate consecutive encoded frames in the first encoded data. Yes. Also, (a ′), (b ′), (c ′), and (d) show configurations at the time of re-encoding a video frame obtained by decoding the encoded data. The frames shown in FIGS. 18A and 18B are interlaced frames. In this case, the re-encoded frames (a ′) and (b ′) form a frame with fields in which 0 and 2 which are the top fields of (a) and (b) are repeatedly displayed.

  That is, the decoded video field is in the field order of 0 → 1 → 2 → 3, but the actual analog display and re-encoded field is configured in the order of 0 → 0 → 2 → 2. At this time, if the first encoded data is subjected to motion compensation prediction from field 0 in field 2 in FIG. 18A, the second encoding is performed using the motion vector 220 in the first encoding. Motion-compensated predictive coding is performed in units of frame macroblocks. That is, the motion vector 223 used at the time of re-encoding in FIG. 18 is obtained by converting a motion vector with 220 fields as a unit into a motion vector with a frame as it is. In the first encoding, when the field 2 is encoded by the motion compensated prediction from the field 1, that is, when the encoding is performed using the motion vector 221 in FIG. Is scaled twice in the vertical and horizontal directions to generate a motion vector 223 for re-encoding. As a result, it is possible to convert the motion vector 221 indicating the amount of motion at the time of one field interval into the motion vector 223 indicating the amount of motion at one frame interval.

  In addition, as shown in FIGS. 18C and 18D, when the first encoded data is a frame image that is not interlaced (progressive image) and motion compensation is performed using a frame macroblock, re-encoding is performed. In the encoding, the motion vector 222 of the first encoded data is used as it is as the motion vector 224 at the time of re-encoding, and the field configuration is re-encoded in the same manner as 0, 1, 2, 3 of the first encoded data. Do.

  Further, in the case of constructing a frame by repeating the same field to prevent jerky movement as in each of the special reproduction processes described above, since the vertical correlation in the frame becomes high, the DCT type always uses a frame block as a luminance block. It is possible to improve the encoding efficiency by selecting the DCT.

[Third Embodiment]
FIG. 19 is a diagram showing a system configuration of a moving image re-encoding device according to the third embodiment of the present invention. In the figure, the components given the same reference numerals as those in FIGS. 1 and 2 are the same as the components in FIGS.

  The characteristic difference between the present embodiment and the first and second embodiments is that the re-encoding unit 28 receives the first encoded data 150 and performs conversion processing, and the video encoder 22. Is added to the selector 152 for selecting the encoded data output from the first encoding data and the first encoded data after the conversion processing from the conversion processing unit 151, and the output of the selector 152 is the moving image data obtained by re-encoding. It is the point which becomes the composition outputted as.

  That is, in this embodiment, when analog output is performed without performing any processing on the video signal obtained by decoding the first encoded data, it is not encoded data obtained by re-encoding the video signal, The corresponding first encoded data is output as it is, and when the video signal obtained by decoding is processed for overlay processing or special reproduction, the video signal after this processing is replayed. The encoded encoded data is selected and output.

  In addition, switching of encoded data to be output is performed in units of frames or macro blocks. When the macroblock unit is switched, if the encoding target macroblock and the reference image area of motion compensation prediction of the macroblock are not affected by the overlay process, the corresponding first encoded data is output as it is, and otherwise In this case, the moving image data obtained by re-encoding is selected.

  Furthermore, in the special reproduction processing shown in FIGS. 13 to 17, when the frame is not generated by repeating the same field, the first encoded data is used instead of the re-encoded moving image data. The data conversion processing unit 151 rewrites data in the header area related to the display order and field configuration described above, and outputs it.

  The flowchart of FIG. 20 shows the operation of the selector 152 in FIG. 19 in the present embodiment. That is, in the case of special playback (trick playback) (Yes in step S1), an uninterlaced frame, that is, a progressive frame (Yes in step S2), an overlay or the like is applied to the decoded video signal. If the pixel processing is not performed (Yes in step S3), or if pixel processing such as overlay is not performed on the video signal decoded by normal reproduction, the first encoded data (stream processing output) is used. In other cases (No in step S2 or No in step S3), the re-encoded encoded data is selected and output (step S5).

(About motion vector scaling according to inter-frame prediction structure)
FIG. 21 shows an example of motion vector scaling according to the inter-frame prediction structure at the time of re-encoding according to the embodiment of the present invention. FIG. 21A shows the inter-frame prediction structure in the first encoded data, and I0, B1, B2, P3,... Represent the I picture, B picture, and P picture of the frame structure in the display order, respectively. ing. FIG. 21B shows an inter-frame prediction structure at the time of re-encoding using an I picture having a frame structure and a P picture having a continuous frame structure. Further, FIG. 21C shows an inter-field prediction structure of re-encoding using an I picture having a field structure and a P picture having a continuous field structure. The arrows indicated by 500 to 521 in the figure indicate the first encoding and re-encoding motion vectors, the starting point of the arrow is the encoding target image, and the end point of the arrow is the reference image.

  In the present embodiment, the first encoded data obtained by encoding the moving image signal with the combination of the I picture, P picture, and B picture having the frame structure of FIG. By re-encoding the moving image signal obtained by the above-described prediction structure shown in FIG. 21B or FIG. 21C, frame rearrangement at the time of re-encoding becomes unnecessary. Therefore, a frame memory for frame rearrangement in re-encoding is not required, and the re-encoding delay amount can be reduced.

  Furthermore, in the example of FIG. 21 (c), since encoding is performed using a field structure picture, it is possible to further reduce the field memory for field / frame conversion and the delay amount during re-encoding. .

  However, since the inter-frame prediction structure is different from the first encoded data in FIGS. 21B and 21C, the motion vectors 500 to 505 extracted from the first encoded data are used as they are when re-encoding. I can't. However, by scaling the motion vector extracted from the first encoded data in consideration of the difference in the prediction structure, it can be used at the time of re-encoding.

  In FIG. 21B, the frames P1, P2, and P3 correspond to B1, B2, and P3 in FIG. 21A, respectively. As the motion vector 510, the motion vector 500 of the first encoded data is used as it is. Can be used. In addition, as the motion vectors 511 and 512, the motion vectors 501 and 502 of the first encoded data can be scaled to 1/2 times and 1/3 times, respectively.

  Further, in the field structure predictive coding as shown in FIG. 21C, in the case of MPEG2 coding, either the immediately preceding in-phase field or reversed-phase field can be used as a reference image. Here, i0 and p1 correspond to the fields constituting the frame I0 in FIG. 21A, and p2 and p3 correspond to the fields constituting B1 in FIG. 21B. Therefore, the motion vector 500 of the first encoded data can be used as the motion vector 520, and the motion vector 500 of the first encoded data is scaled to ½ times as the motion vector 521. Can be used.

  That is, in order to reduce the cost and delay time of the re-encoding unit, even when the inter-frame prediction structure at the time of re-encoding is different from the first encoded data, the motion vector extracted from the first encoded data Is scaled in consideration of the difference in the prediction structure, it is possible to perform re-encoding by predictive encoding using motion compensated prediction without newly detecting a motion vector at the time of re-encoding.

(About sub-picture signal or GUI image display switching)
Next, a display switching method for sub-video signals or GUI images according to the embodiment of the present invention will be described.
As a technique for displaying a movie with a frame rate of 24 Hz as an interlaced image with a standard television frame rate of 29.97 Hz, there is a technique called 3: 2 pull-down. 3: 2 pull-down makes it possible to display a frame of a movie at a frame rate of 29.97 Hz by alternately switching between displaying one frame of a movie for each frame for three field periods or two field periods. In the display of the three-field period, the video displayed in the third field is a display in which the first field in the same frame is repeatedly displayed. In MPEG2 encoding, encoding corresponding to 3: 2 pull-down is possible. In other words, in the encoding of movies and the like, encoding is performed in units of frame images with a frame rate of 24 Hz, and the encoded frame is encoded as a frame that is displayed for a period of 3 fields or a frame that is displayed for a period of 2 fields. This is indicated by a flag in the data, and is converted from a coding frame rate of 24 Hz to a display frame rate of 29.97 Hz in the decoding device.

  FIG. 22 is a diagram showing an example of display switching of the sub-video signal or GUI image according to the embodiment of the present invention. FIG. 22A shows display field numbers, and FIG. 22B shows the structure of an encoded frame corresponding to 3: 2 pull-down processing. In the example of FIG. 22, the frame composed of fields 0 and 1 is the first encoded frame, and the frame composed of fields 2, 3, and 4 is the second encoded frame. Here, field 4 is deleted at the time of encoding, and after decoding, field 2 is repeatedly displayed at the timing of field 4. A field indicated by a dotted line in FIG. 22B is a field repeated by 3: 2 pull-down, and FIG. 22C shows a field configuration after decoding.

  FIG. 22D shows the switching timing of the sub-video signal or GUI image. In this example, the first sub-video or GUI image is displayed statically from field 0 to field 7, and from field 8. Up to field 16, the second sub-picture or GUI image is displayed statically. Here, the switching of the sub-video signal or the GUI image signal is a mixing ratio when the sub-video or the GUI image is faded and displayed by turning on / off the display itself or changing the mixing ratio with the video signal stepwise. It means the change point of

  In the example of FIG. 22D, the first sub-video or GUI image is displayed in an overlay manner at the time of the field 7, and the second sub-video or GUI image is displayed as an overlay at the time of the field 9. Therefore, the video of the field 9 that originally displays the same video as the video of the field 7 does not match the video of the time of the field 7 after the overlay processing.

  When the video signal after the overlay processing is re-encoded with the same frame configuration as the first encoded data, that is, when the video at the time of field 9 is regarded as a repetition of the video of field 7, When the encoded data obtained by re-encoding is decoded, the displayed sub-video or GUI image is displayed in the order shown in FIG. 22 (e), and the video is accompanied by an unnatural time change. Will be disturbed.

  On the other hand, as shown in FIG. 22 (f) or (g), if the change point of the sub-picture signal or GUI image is shifted to the boundary of the field constituting the encoded frame shown in FIG. Even if the encoded video signal is re-encoded with the same frame configuration as the first encoded data, the sub-video signal or the GUI image is temporally disturbed when the video signal obtained by decoding the re-encoded data is displayed. It is possible to display a smooth image without occurrence.

1 is a block diagram showing the configuration of a system including a moving image re-encoding device according to a first embodiment of the present invention. FIG. 3 is a block diagram showing a configuration example of a video encoder for re-encoding in the embodiment Diagram showing an example of a graphics overlay image The figure which shows the example of operation timing of the moving image re-encoding apparatus which concerns on the same embodiment The block diagram which shows the structure of the system containing the moving image re-encoding apparatus which concerns on the 2nd Embodiment of this invention. The figure which shows the example of operation timing of the moving image re-encoding apparatus which concerns on the same embodiment Diagram explaining DCT type related to MPEG Diagram showing an example of a graphics overlay image Diagram showing an example of a graphics overlay image The block diagram which shows the other structural example of the video encoder for the re-encoding in this invention Diagram showing an example of a graphics overlay image The figure which shows the example of the motion vector scaling accompanying frame rate conversion The figure which shows the example of the fast-forward reproduction | regeneration which concerns on this invention The figure which shows the example of the fast-forward reverse play based on this invention The figure which shows the example of the pause reproduction | regeneration based on this invention The figure which shows the example of the fast-forward reproduction | regeneration which concerns on this invention The figure which shows the example of the slow reproduction which concerns on this invention The figure which shows the example of the motion vector scaling concerning this invention The block diagram which shows the structure of the system containing the moving image re-encoding apparatus which concerns on the 3rd Embodiment of this invention. The figure which shows the control flowchart which concerns on the 2nd Embodiment of this invention. The figure which shows the example of the motion vector scaling which concerns on embodiment of this invention The figure which shows the example of the display switching of a sub video signal or a GUI image which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Data separation part 11 ... Audio | voice data buffer memory 12 ... Moving image data buffer memory 13 ... Sub video data buffer memory 14 ... Navigation data buffer memory 15 ... Audio decoder 16 ... Video decoder 17 ... Sub video decoder 18 ... Video overlay processing part DESCRIPTION OF SYMBOLS 20 ... GUI image signal generation part 21 ... Audio encoder 22 ... Video encoder 23 ... Frame memory 24 ... Audio | voice data buffer memory 25 ... Moving image data buffer memory 26 ... Multiplexer 27 ... IEEE1394 interface 28 ... Re-encoding part 29 ... Audio | voice Signal D / A Converter 30 ... Video Signal D / A Converter 50 ... Optical Disc Device 67 ... Video Signal Selector 70 ... Motion Compensation Unit 72 ... Motion Compensation Mode and DCT Type Determination Unit 73 ... Rate Control Unit 74 ... DC T and quantization unit 75 ... inverse quantization and inverse DCT unit 76 ... variable length coding unit 151 ... moving image data conversion unit 152 ... moving image coded data selector

Claims (7)

Decoding means for decoding a first encoded data obtained by performing motion compensation prediction encoding of a moving image signal to obtain a moving image signal;
Coding parameter extraction means for extracting a coding parameter including at least a motion vector and coding mode information from the first coded data;
Means for generating a sub-video signal or GUI image signal to be overlaid on a video frame or video field of the video signal obtained by the decoding means;
Means for sequentially performing overlay processing of the sub-video signal or the GUI image signal on the video frame or video field;
Re-encoding means for re-encoding the video frame or video field subjected to the overlay processing to obtain second encoded data;
The re-encoding means is (a) a magnitude of a motion compensation prediction error in the first motion compensation prediction encoding mode using a motion vector in the extracted encoding parameter, and (b) given from the outside. A magnitude of motion compensation prediction error in the second motion compensation prediction coding mode using a motion vector determined from a display position of the sub-picture signal or the GUI image signal and a motion amount thereof; and (c) intra coding. The signal size of the encoding target macroblock in the mode is compared, and according to the comparison result, any one of the first motion compensation prediction encoding mode, the second motion compensation prediction encoding mode, and the intra encoding mode is selected. An encoding mode selection unit that selects an encoding mode for each macroblock, and performs the re-encoding according to the selected encoding mode; Video transcoder according to claim.   The re-encoding unit further includes a storage unit that temporarily stores the encoding parameter extracted by the encoding parameter extraction unit, and the storage unit stores the storage unit in the first motion compensation prediction encoding mode. The moving image re-encoding apparatus according to claim 1, wherein the re-encoding is performed using a motion vector in an encoding parameter.   The means for generating the sub video signal or the GUI image signal generates the sub video signal or the GUI image signal corresponding to the decoding order of the video frame or video field of the moving image signal obtained by the decoding means. 2. The moving image re-encoding apparatus according to claim 1, wherein the re-encoding unit re-encodes the video frame or the video field subjected to the overlay processing in accordance with the decoding order. The encoding parameter extraction means further extracts a frame / field adaptive orthogonal transform type as the encoding parameter from the first encoded data,
The re-encoding unit includes a frame / field adaptive orthogonal transform unit, and (a) a macroblock in which an encoding mode using the same encoding parameter as the encoding parameter is selected by the encoding mode selection unit For the frame / field adaptive orthogonal transform type extracted from the first encoded data, and (b) motion compensated predictive encoding using a predetermined motion vector by the encoding mode selection means, and (c ) For a macroblock for which any coding mode of intra coding is selected, an optimum frame / field adaptive orthogonal transform type is selected, and orthogonal transform is performed by the frame / field adaptive orthogonal transform means. The moving image re-encoding device according to any one of claims 1 to 3. Decoding means for decoding a first encoded data obtained by performing motion compensation prediction encoding of a moving image signal to obtain a moving image signal;
Coding parameter extraction means for extracting a coding parameter including at least a motion vector and coding mode information from the first coded data;
Means for generating a sub-video signal or GUI image signal to be overlaid on the video frame or video field of the video signal obtained by the decoding means;
Means for sequentially performing overlay processing of the sub-video signal or the GUI image signal on the video frame or video field;
Re-encoding means for re-encoding the video frame or video field subjected to the overlay processing to obtain second encoded data;
The re-encoding means is selected when an overlay ratio of the sub-video signal or the GUI image signal to the video frame or video field is less than a threshold value. (A) The same code as the extracted encoding parameter (B) Motion compensation in motion compensated prediction coding mode using a predetermined motion vector selected when the size of the motion compensated prediction signal in the coding mode using the encoding parameter and the overlay ratio are equal to or greater than the threshold value The magnitude of the prediction error signal and (c) the magnitude of the signal of the macroblock to be coded in the intra coding mode are compared, and the coding mode of (a) or the motion compensated prediction code of (b) is compared according to the comparison result. One of the coding mode and the intra coding mode (c) is selected for each macroblock. It has a coding mode selecting means, the moving picture re-encoding apparatus and performs the re-encoding according to the selected coding mode. The re-encoding means includes data extraction means for extracting encoded data in units of frames or macroblocks from the first encoded data, and the first encoded data is fast-forwarded, thrown, top-fed, and paused. When special reproduction processing or frame rate conversion processing including at least one has been performed, the encoded data extracted by the data extraction means includes header data including at least data related to the display order and display field configuration. Changing the header data changing means to obtain third encoded data that satisfies the condition that the top field and the bottom field are alternately displayed even when the frame is repeatedly displayed by the special reproduction processing; Replacement means for replacing a part of the encoded data with the third encoded data Have,
The encoded data replacement means selects and outputs the second encoded data at the time of special reproduction including at least one of fast-forward, slow, top-feed, and pause of an interlaced video signal in the playback device, In the normal reproduction with frame rate conversion from 24 Hz to 29.97 Hz in the reproduction apparatus and special reproduction including at least one of fast-forwarding, slowing, top-feeding, and pause of the progressive video signal, the third code 6. The moving image re-encoding apparatus according to claim 1, wherein the encoded data is selected and output. Means for shifting the timing of changing the sub-video signal or the GUI image signal to be subjected to the overlay processing to the boundary of the decoded video frame or video field corresponding to the encoded frame in the first encoded data The moving image re-encoding device according to claim 1, further comprising:

Images