JP2000244929A - Moving picture re-encoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a moving picture re-encoding device at a low cost that has a high degree of freedom for special reproduction and overlay processing or the like to attain digital connection between digital video systems adopting different methods and provides an encoded picture with high image quality. SOLUTION: A video decoder 16 decodes moving picture coded data 52 reproduced from an optical disk 50 to reproduce a moving picture signal 56 and to extract coding parameters including information such as a motion vector, a coding mode and a quantization step from the moving picture coded data 52, and a re-encoding section 28 applies re-encoding to the moving picture signal 56 according to the extracted coding parameter in this moving picture re- encoder. The re-encoding section 28 selects any coding mode for each macro block from among (a) a coding mode where the same coding parameters as for the extracted coding parameters are used, (b) a compensation prediction coding mode using an existing motion vector 65, and (c) an intra-coding mode.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture re-encoding device, and more particularly to a method for decoding moving image data obtained by encoding according to the MPEG2 system or the like, and for directly or graphics overlaying the obtained moving image signal. The present invention relates to a moving image re-encoding device that performs re-encoding after appropriately performing a special reproduction process or the like.

[0002]

2. Description of the Related Art In a reproducing apparatus of a digital video system such as a DVD (digital versatile disc or digital video disc) system, compressed moving picture data, audio data, sub-picture data and the like obtained from a reproduced signal are multiplexed. Separation and decoding of each multiplexed data are performed. A moving image signal (video signal) obtained by decoding the moving image data is subjected to overlay processing of a sub-video or a GUI (graphical user interface) specific to a playback device, and then D / A converted and output as an analog signal. You. An audio signal obtained by decoding audio data is D / A converted and output as an analog signal, or is output as a digital signal.

In recent years, digital interfaces in homes such as IEEE 1394 have been standardized, and digital TVs for digital broadcasting have been developed. With the digitalization of the video equipment in the home and the interface between the equipment, the video signal transmission between the video equipment in the home, which has conventionally relied on the analog signal, can be digitized.

[0004] On the other hand, there are various standards for digital video and digital audio coding systems for each application, and in fact, compatibility between these standards is not sufficiently ensured. For example, a DVD recording format, a format for a digital video camera such as a DVCR (Digital Video Cassette Recorder),
None of the digital broadcasting formats and the like have compatibility. For digital broadcasting, the US system (A
TSC), the European system (DVB), the Japanese system, etc., have different systems for each region, and compatibility of digital data between these systems cannot be guaranteed. Therefore, in order to connect various digital video devices with a digital interface, it is necessary to perform conversion of a data format including conversion of an encoding method according to each application.

[0005] Data format conversion methods can be broadly classified into the following two methods. The first conversion method is a data format conversion process at a digital data level which is encoded data. In this method, the degree of freedom of conversion may be greatly limited depending on the coding method of the coded data before conversion. For example, performing digital processing such as special reproduction, effect processing, or video mixing on moving image data obtained by encoding in the MPEG2 system involves very large restrictions. Also, conversion at a digital data level to a different encoding scheme is often difficult to achieve.

A second data format conversion method is as follows.
The encoded data is decoded into a baseband signal, and the baseband signal is encoded again by a desired encoding method (this is referred to as re-encoding). In this case, the degree of freedom of the data format conversion is large, and digital reproduction such as special reproduction, effect processing, or video mixing can be performed on the baseband signal, and there is no restriction. However, since many coding methods are irreversible conversions, in this method, image quality and sound quality are degraded every time the conversion is repeated. In addition, in order to perform high-efficiency re-encoding, it is necessary to implement an additional encoding device in digital video equipment, and in particular, MPEG2
The addition of an encoding device with motion compensation such as an encoder causes a significant increase in the cost of each digital video device.

[0007]

As described above, in order to digitally connect digital video equipment having different data formats, conversion of a data format including conversion of an encoding system is performed by using conventional encoded data ( The method of performing conversion at the (digital data) level has a problem that the degree of freedom of processing such as special reproduction, effect processing, and video mixing is small, and special reproduction and effect processing are performed at the baseband signal level by decoding encoded data. However, the method of performing re-encoding with a desired encoding method by performing processing such as video mixing has a problem that image quality is deteriorated and the cost of digital video equipment is increased.

The present invention has a degree of freedom for special reproduction and other processing by processing at a baseband signal level, suppresses image quality deterioration due to data format conversion including an encoding method, and further performs additional re-encoding. It is an object of the present invention to provide a moving picture re-encoding device capable of greatly reducing the cost of video.

[0009]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention provides fast-forward, slow, pause, etc., for a moving image signal which is a baseband signal obtained by decoding encoded moving image data. , Special playback processing, sub-picture or GUI overlay processing, etc., and furthermore, an optimal code based on information of various coding parameters extracted from the moving picture coded data at the time of decoding, special reproduction and overlay processing, etc. In this case, the encoding parameters are determined and low-cost and high-quality re-encoding can be realized.

That is, a first moving picture re-encoding device according to the present invention decodes first coded data obtained by motion-compensated predictive coding of a moving image signal to obtain a moving image signal. Means, coding parameter extracting means for extracting, from the first coded data, coding parameters including at least information of a motion vector, a coding mode and a quantization step, and a moving picture obtained by the decoding means. Re-encoding means for re-encoding the signal in accordance with the encoding parameter extracted by the encoding parameter extracting means to obtain second encoded data, wherein the re-encoding means comprises:
One of (a) encoding using the same encoding parameter as the extracted encoding parameter, (b) motion-compensated predictive encoding using a predetermined motion vector, and (c) intra-encoding A coding mode selection unit for selecting one coding mode for each macroblock.

In the first moving picture re-encoding device, a motion vector detecting circuit which requires a large number of resources in a normal encoding device is not required in re-encoding, so that a significant cost reduction can be realized. Becomes

In a region where no processing is applied to a video signal which is a moving image signal obtained by decoding,
Basically, (a) is automatically selected as the encoding mode of the re-encoding, and the first encoded data is made to coincide with all the encoding parameters, thereby minimizing the image quality deterioration accompanying the re-encoding. It can be suppressed. On the other hand, in a region where some processing has been performed on the video signal, predictive coding using a predetermined motion vector of (b) or intra-frame coding of (c) is selected as a coding mode of re-coding. In some cases. By setting the value of the predetermined motion vector according to the processing content of the video signal, it is possible to perform predictive coding with high coding efficiency without requiring a motion vector detection circuit.

A second moving picture re-encoding device according to the present invention provides a sub-picture signal or a GUI (Graphical User Interface) picture corresponding to the picture frame or picture field of the moving picture signal obtained by the decoding means. Further comprising: means for generating a signal; and means for sequentially performing overlay processing of the sub-video signal or the GUI image signal on the video frame or the video field; Storage means for temporarily storing the encoding parameters extracted by the means,
The video frame or the video field on which the overlay processing has been performed is re-encoded by using the encoding parameter stored in the storage unit.

In moving picture coding such as MPEG2, there are forward and backward predictions between frames, so that the order of encoding and decoding generally does not match the order of display frames. These differences in the frame order are absorbed by the frame rearrangement process in each of the encoding device and the decoding device. In a system in which graphics overlays and the like are performed in the display order, it is difficult to re-encode at the timing of decoding. In this case, since re-encoding is performed after decoding and graphic overlay processing,
There is a certain delay between the constant decoding and the re-encoding. However, by temporarily storing the extracted encoding parameters and delaying the extracted encoding parameters in the same manner, re-encoding using the same encoding parameters as the first encoded data becomes possible.

A third moving picture re-encoding device according to the present invention is the first moving picture re-encoding device, wherein the decoding order of the video frame or the video field of the moving image signal obtained by said decoding means is determined. Corresponding to the sub-video signal or G
Means for generating a UI image signal, and the sub-video signal or the G
Means for sequentially performing overlay processing of a UI image signal, wherein the re-encoding means re-encodes the video frame or video field on which the overlay processing has been performed in accordance with the decoding order. .

As described above, due to the difference in the frame order between the encoding of MPEG2 and the like and the display, a delay occurs due to frame rearrangement in decoding and re-encoding, and normal encoding is performed for re-encoding. Requires a frame memory for rearranging frames. On the other hand, in the third moving picture re-encoding device according to the present invention, in the decoding order, the sub-picture and the GUI overlay processing are performed almost simultaneously with the decoding, and the re-encoding processing is further performed by decoding and decoding. Since it can be performed simultaneously with the overlay processing, the delay amount due to the frame rearrangement can be minimized, and a configuration that does not require a memory for the frame rearrangement in the re-encoding means is adopted. It becomes possible. The re-encoding means only needs to have a frame memory for reference images for predictive encoding.

According to a fourth aspect of the present invention, in the above-described first to third moving image re-encoding devices, the encoding parameter extracting means includes the first encoded data from the first encoded data. A frame / field adaptive orthogonal transform type is further extracted as an encoding parameter, and the re-encoding means has a frame / field adaptive orthogonal transform means, and (a) the same as the encoding parameter is selected by the encoding mode selecting means. For a macroblock for which a coding mode using a coding parameter has been selected, a frame / field adaptive orthogonal transform type extracted from the first coded data is selected, and (b) a default is selected by the coding mode selecting means. Motion compensation predictive coding using the motion vector of B The block by selecting an optimal frame / field adaptive orthogonal transform type, and characterized by applying orthogonal transformation respectively by the frame / field adaptive orthogonal transform means.

As described above, in an area where no processing has been applied to the video signal, the first encoded data and all the encoding parameters are matched to minimize the image quality deterioration due to re-encoding. It can be suppressed. However, it is not always optimal to match the first encoded data with all encoding parameters in a portion where the video signal has been subjected to special reproduction, overlay processing, or the like. In such a portion, the frequency at which the predictive coding using the predetermined motion vector (b) or the intra coding (c) is automatically selected increases. Accordingly, in the macroblock for which (b) or (c) is selected, the optimal type is reselected for the field / frame adaptive orthogonal transform (DCT (discrete cosine transform) in MPEG2 or the like), and the selected type is re-selected. By performing the encoding, the encoding efficiency can be improved.

A fifth moving picture re-encoding apparatus according to the present invention is characterized in that, in the first to fourth moving picture re-encoding apparatuses, in particular, the predetermined motion vector of (b) is a zero vector. I have.

In special playback such as fast-forward, slow-movement, and frame-by-frame playback, the video of the same frame may be displayed a plurality of times. In many cases, sub-pictures are displayed at the same position. In any case, by using motion compensation using a zero vector, efficient prediction coding can be realized without a motion vector detection circuit.

According to a sixth moving picture re-encoding device of the present invention, in the first to fourth moving picture re-encoding devices, in particular, the predetermined motion vector of (b) may be the sub-video signal or the GUI image. Are determined from the display position and the amount of movement.

In advanced sub-video and GUI images overlaid on video signals, it may be possible to move a still image temporally within a video frame. In the sixth moving image re-encoding device, it is possible to calculate an optimal motion vector from the amount of motion of an image to be overlaid according to the macroblock position of the overlaid area, and the default motion vector shown in FIG. By setting the motion vector value to the optimal value for the value of, the encoding efficiency can be improved.

According to a seventh moving picture re-encoding device according to the present invention, in the first to fourth moving picture re-encoding devices, the encoding mode selecting means may select the sub-video signal or the GU.
The encoding mode is selected by using at least one of a display position of an I image signal and a motion amount thereof, an area ratio of each moving image signal obtained by the decoding means with respect to each macroblock, and a mixing ratio. And

In the first to fourth moving picture re-encoding devices,
(A) coding using the same coding parameter as the above coding parameter, (b) motion compensation predictive coding using a predetermined motion vector, and (c) intra coding selected for each macroblock For example, the motion compensation prediction error power in the coding of (a), the motion compensation prediction error power in the coding of (b), the signal power of the coding target macroblock in the coding of (c), and the like are calculated. ,
It is necessary to determine the optimal mode by comparing each.

Here, in the sixth moving picture re-encoding device,
The comparison between the coding modes (a) and (b) is performed using the overlay ratio instead of the error power, and the selected coding mode is compared with the intra coding (c) using the prediction error power or the like. Configuration. When the overlay ratio is determined by mixing the area ratio of the overlay pixel in the macroblock and the video signal,
It is determined from the mixing rate, etc. If the overlay ratio is equal to or larger than the threshold, the mode of (b) may be selected, and if not, the mode of (a) may be selected. This makes it possible to reduce the number of errors and signal power calculations from three to two for each macroblock.

According to an eighth moving picture re-encoding device according to the present invention, in the first to fourth moving picture re-encoding devices, the re-encoding means converts the moving picture signal obtained by the decoding means. A resolution conversion unit that performs resolution conversion for enlargement or reduction on at least a part thereof, and a motion vector extracted from the first encoded data or the predetermined motion vector according to a conversion ratio of the resolution conversion unit. Scaling means for performing scaling, wherein the coding mode selecting means selects the coding mode using the motion vector after the scaling.

When a video signal is re-encoded and output after resolution conversion after decoding, the motion vector and encoding parameters of the first encoded data cannot be used as they are. However, by appropriately scaling the motion vector information according to the resolution conversion ratio, it is possible to obtain a substantially appropriate motion vector without performing motion vector detection during re-encoding.

According to a ninth moving picture re-encoding device according to the present invention, in the first to fourth moving picture re-encoding devices, the re-encoding means converts the moving picture signal obtained by the decoding means. Rate conversion means for converting a display frame rate or a display field rate for at least a part thereof;
Scaling means for scaling a motion vector extracted from the first encoded data or the predetermined motion vector in accordance with a conversion ratio of the rate conversion, wherein the encoding mode selecting means determines that the scaling is The coding mode is selected using the motion vector after the coding.

If the decoded video signal is re-encoded and output after conversion of the frame frequency or field frequency by thinning out a frame or a field, repeating display, interpolation, etc., the first code If the motion vector and the encoding parameter extracted from the encoded data are used as they are, image quality may be degraded. By performing a conversion process including scaling on the extracted motion vector in accordance with the conversion of the frame or the field frequency, it is possible to suppress the image quality deterioration.

According to a tenth moving picture re-encoding device of the present invention, in the first to fourth moving picture re-encoding devices, the re-encoding means has fast forward, slow,
A motion vector conversion unit configured to convert a motion vector extracted from the first encoded data in accordance with a reproduction parameter related to a special reproduction function including at least one of frame forwarding and pause; The encoding mode selection means selects the encoding mode using the motion vector converted by the motion vector conversion means.

At the time of trick play, one field of the interlaced image is deleted and the remaining fields are displayed continuously, the frames or fields to be decoded are thinned out, or the decoded frames or fields are displayed a plurality of times. Is performed. In the tenth moving image re-encoding device, by performing the conversion of the motion vector in accordance with the special reproduction processing, it is possible to suppress image quality deterioration when re-encoding the special reproduction image.

According to an eleventh moving picture re-encoding device according to the present invention, in the first to fourth moving picture re-encoding devices, the re-encoding means converts a frame or a macro block from the first encoded data. Data extracting means for extracting encoded data in units, and header data changing for changing predetermined header data of the encoded data extracted by the data extracting means, the data including at least data relating to the display order and the configuration of display fields Means, and data replacing means for replacing a part of the second encoded data with encoded data whose header data has been changed by the data extracting means and the header data has been changed by the header data changing means. I do.

For each of the coded image frame and the reference image frame related to the coded frame, the frame or region to be re-encoded without any processing after decoding is the same as the first coded data. By performing the re-encoding using the encoding parameters, the deterioration of the image quality due to the re-encoding becomes sufficiently small. However, in some cases, complete reversibility may not be achieved due to the effects of rounding, limiter processing, and the like due to the calculation accuracy in the decoding process or the re-encoding process.

In the eleventh moving picture re-encoding device, the region of the re-encoded second coded data in which the video signal has not been subjected to any processing as described above corresponds to the corresponding first coded data. By replacing the encoded data with the encoded data and outputting the encoded data, it is possible to completely remove even a slight deterioration in image quality due to the calculation accuracy. If special reproduction processing, frame rate conversion, or other processing has been performed, the first encoded data is output as necessary to satisfy the validity of the encoded data to be output.
The first encoded data can be output as re-encoded data by appropriately changing the value of the predetermined data indicating the display order and the display field configuration of the header area among the encoded data. .

According to a twelfth moving picture re-encoding device according to the present invention, in the eleventh moving picture re-encoding device, the coded data replacing means includes a motion vector extracted from the first coded data and , The sub-video signal or the GU
It is characterized in that coded data is replaced in units of macroblocks based on the display position of the I image signal and the amount of movement thereof.

In the twelfth moving picture re-encoding device, it is possible to specify both the area to be overlaid and the area to be referred to in the motion compensation prediction coding by each macroblock of the decoded moving picture. It is. From these pieces of information, it is possible to determine macroblocks affected by overlay processing and macroblocks not affected by overlay processing, and to output the first encoded data as it is or to re-encode the second encoded data. Whether to output data can be switched for each macroblock and re-encoded and output.

According to a thirteenth moving picture re-encoding device of the present invention, in the eleventh moving picture re-encoding device, the encoded data replacing means may include a fast-forward, a slow-motion, a frame-forward and a pause of the interlaced moving image signal. At the time of special reproduction including at least one of the following, the second encoded data is selected and output, and at the time of normal reproduction and special reproduction of a progressive moving image signal, the second encoded data is extracted by the data extracting means and the header data changing means. The encoded data whose header data has been changed is selected and output.

Here, the interlaced video signal is
The video of the even-numbered line and the odd-numbered line in the frame is a video at a time separated by half the frame period, and the progressive video signal is a video of the even-numbered line and the odd-numbered line in the frame at the same time. It is an image.

In slow reproduction or frame feed, processing for repeatedly displaying the same video frame is performed. Also in fast-forward playback, frame thinning and repetitive display of the same video frame may be performed in combination depending on the transfer rate of encoded data. As described above, when the same video frame is repeatedly displayed, when displaying a moving video in an interlaced frame, the display has a jerky motion because the video has motion in even and odd lines. Will be. Normally, a process for performing smooth display is performed by selecting and repeatedly displaying either the even-numbered line or the odd-numbered line. However, it is difficult to extract only the video of one line (that is, the field) at the coded data level from the motion compensated prediction coded data of the interlaced moving image signal.

In such a case, smooth display can be realized by outputting the second encoded data obtained by encoding the baseband video signal after the special reproduction processing according to the present invention. Becomes Also, since there is no video motion in a frame in a progressive image,
Special reproduction can be realized by rearranging the first encoded data, changing and correcting the header data, and the like. By adaptively combining the two and outputting, it is possible to minimize image quality deterioration due to re-encoding and prevent image disturbance during special reproduction.

According to a fourteenth moving picture re-encoding device according to the present invention, in the first to fourth moving picture re-encoding devices, the re-encoding means may include an interframe different from the first encoded data. Means for performing re-encoding with a prediction structure, wherein the means for performing scaling on a motion vector extracted from the first encoded data according to an inter-frame prediction structure at the time of the re-encoding, The encoding mode selection means selects the encoding mode using the scaled motion vector.

As described above, in order to reduce the cost and delay time of the re-encoding unit, even if the inter-frame prediction structure at the time of re-encoding is made different from that of the first coded data, Re-encoding by predictive coding using motion-compensated prediction without re-detecting a motion vector at the time of re-encoding by scaling the motion vector extracted from It becomes possible.

According to a fifth moving picture re-encoding device of the present invention, in the first to fourth moving picture re-encoding devices, the decoding corresponding to the encoded frame in the first encoded data is performed. Means for switching the display of the sub-video signal or the GUI image signal to be subjected to the overlay processing in accordance with the boundary of the displayed video frame or video field.

When the video signal after the overlay processing is re-encoded in the same frame configuration as the first encoded data, when the encoded data obtained by the re-encoding is decoded,
In some cases, the displayed sub-video or GUI image may cause video disturbance with an unnatural time change.

The fifteenth moving picture re-encoding device shifts the sub video signal or the change point of the GUL image to the boundary between the fields constituting the coded frame, thereby converting the overlay-processed video signal into the first code. Even if re-encoding is performed with the same frame configuration as the encoded data, the sub-video signal and the GU1 image are not temporally disturbed when displaying the decoded video signal of the re-encoded data, and the Can be displayed.

[0046]

Embodiments of the present invention will be described below with reference to the drawings. [First Embodiment] FIG. 1 shows the configuration of a system including a video transcoder according to a first embodiment of the present invention. This system has a configuration in which a conventional digital video / audio reproducing apparatus such as a DVD player has a digital output interface to an IEEE 1394 bus in addition to analog video and analog audio output. here,
A portion surrounded by a broken line 28 in the figure is a re-encoding unit added to the conventional digital video / audio reproducing apparatus.

The optical disk 50 is, for example, a DVD disk, in which audio data, encoded moving image data, sub-video data, navigation data, and the like are multiplexed and recorded. A signal reproduced from the optical disk 50 is input to the data separator 10, and audio data 51, moving image data 52, sub-video data 53, navigation data 5
4 are separated. Each of the separated data 51,
52, 53, 54 are buffer memories 11, 12, 1
Audio decoder 1 via
5, decoded by the video decoder 16 and the sub-picture decoder 17.

A video signal 56 obtained by decoding the video data by the video decoder 16 and a sub video signal 57 obtained by decoding the sub video data by the sub (Graphical User In
a GUI image signal 59 independently generated by the image generator 20 in response to the user operation 60 and a video overlay processing unit (OSD) via the adder 19
An overlay (synthesis) process is performed by the control unit.

FIG. 3 shows an example of an overlay image for a moving image signal 56 obtained by decoding. FIG. 3A shows a frame (moving image frame) of the moving image signal 56,
FIG. 3B shows a moving image frame after the overlay. The subtitle portion 111 in FIG. 3B is a sub video signal encoded as a bitmap video, is decoded by the sub video decoder 17, and is called a sub picture in the DVD system. The time display 110 in FIG. 3B is obtained by overlaying a time that is not recorded on the optical disc 50 but is generated by the reproducing apparatus itself as a graphic on a video.
Returning to FIG. 1, the audio signal 55 obtained by decoding the audio data by the audio decoder 15 and the moving image signal 58 subjected to the overlay processing by the video overlay processing unit 18
Are converted into analog signals 61 and 62 by D / A converters 29 and 30, respectively, and output to the outside. further,
The audio signal 55 and the moving image signal 58 after the overlay processing are re-encoded by the audio encoder 21 and the video encoder 22, respectively. The audio data and the moving image data obtained by the re-encoding are multiplexed into a transport stream by a multiplexing unit 26 via buffer memories 24 and 25, respectively, and further multiplexed.
It is digitally output from the EEE1394 interface 27.

From the video decoder 16, at the same time as the decoding, the coding parameters 64 included in the moving picture data 52 before decoding, such as the motion vector, coding mode, and quantization step for each macroblock, are extracted. This encoding parameter 64 is temporarily stored in the data memory 31.
The video encoder 22 reads, from the coding parameter data temporarily stored in the data memory 31, coding parameters corresponding to an image frame and a macroblock to be re-encoded, and outputs predetermined motion vector information externally provided. 65 is input.

(Second Configuration Example of Video Encoder 22)
FIG. 2 is a diagram showing a first configuration example of the video encoder 22 for re-encoding in FIG. In the figure, components assigned the same reference numerals as those in FIG. 1 indicate the same components as those in FIG. This video encoder 22 has an MPE
It is based on a motion compensated prediction orthogonal transform coding scheme represented by G2 or the like, and includes a motion compensation unit 70, a motion compensation mode (coding mode) and DCT type determination unit 72, a DCT and quantization unit 74, an inverse quantization and Inverse DCT unit 75, frame memory 23, rate control unit 73, variable length coding unit 7
6 is comprised.

This video encoder 22 is a normal MP
It is characterized in that there is no motion vector detection circuit necessary for a moving picture encoding device such as an EG2 encoder. An ordinary moving picture coding apparatus requires an enormous amount of block matching calculation in order to detect a motion vector for each macroblock. Generally, the hardware resources occupied by the motion vector detection circuit in the entire encoding device occupy a very high ratio. However, the video encoder 22 does not need such a motion vector detection unit having a large hardware scale. That is, in the present embodiment, motion compensation is performed using both the motion vector 64 extracted from the video data 52 before decoding and the predetermined motion vector 65 provided from the outside.

Next, the motion compensation mode and DCT type determination section 72 determines which one of the motion vectors 64 and 65 from the power or magnitude of the prediction error signal by the respective motion compensation and the power or magnitude of the image signal to be encoded. Is determined for each macroblock.

Next, for a macroblock for which predictive coding by the motion vector 64 extracted from the video data 52 before decoding is selected, a DCT type (to be described later) is extracted from the DCT type extracted from the video data 52 before decoding. In other cases, the optimum DCT type is detected.

The rate control unit 73 determines a quantization step for each macro block used in the quantization unit 74. That is, for a macroblock for which predictive encoding using the motion vector 64 extracted from the video data 52 before decoding is selected, the quantization extracted from the video data 52 is performed when there is no restriction on the output bit rate. If the step is used as it is and the output bit rate is restricted, the quantization step is determined with reference to the generated code amount 80 so that the desired bit rate is obtained. In addition, when a predetermined motion vector 65 from the outside is selected,
Alternatively, for a macroblock for which intra-coding has been selected, if there is no restriction on the output bit rate, the quantization step extracted from the moving image data 52 or a predetermined quantization step is used to determine the output bit rate. If the rate is restricted, the quantization step is determined with reference to the generated code amount 80 so that the desired bit rate is obtained.

(Operation of Moving Picture Recoding Apparatus) FIG. 4 shows an example of operation timing of the moving picture recoding apparatus according to the present embodiment. Here, it is assumed that both decoding and re-encoding conform to MEPG2. In the figure, I indicates an intra coded picture (I picture), P indicates a forward predictive coded picture (P picture), B indicates a bidirectional predictive coded picture (B picture), and the suffix number of each frame indicates Indicates the order in which they are displayed.

In FIG. 4, reference numeral 101 (Video DT)
S) indicates the timing at which each frame is decoded, and the time is described in the encoded data as DTS (Decoding Time Stamp: decoding time management information).
In MPEG2, the order of encoding and decoding and the order of display are generally different for inter-frame prediction of B pictures. In the example of FIG. 4, reference numeral 101 (Video
DTS is the timing at which the main video of each frame is displayed, and the time is indicated by the PTS (Presentation Time Stamp:
Playback output time management information) is described in the encoded data. In MPEG2, the PTS of a B picture matches the DTS of the same frame, and the PTS of an I picture and a P picture match the DTS of an I picture or a P picture to be decoded next.

In the case of DVD, the sub-picture signal mapped to the main picture signal is decoded in the display order, and its time DTS
Coincides with the sub video display time PTS. Reference numeral 102 (SP DTS) in FIG. 4 indicates the timing of decoding and displaying the sub-video signal corresponding to each moving image frame. The sub video signal is overlaid on the main video signal simultaneously with decoding. Reference numeral 103 (OSD) in FIG. 4 indicates the timing of the overlay processing.

In the example of FIG. 4, the sub-video signal corresponding to each video frame is decoded and displayed as the video signal after overlay, but the sub-video frame exists for all the video frames. The same sub-video signal may be repeatedly displayed for a certain period of time, or processing such as highlighting a part of the sub-video signal in accordance with the navigation processing may be performed.

The output of the analog video signal can be started at the same time as the overlay processing, so that reference numeral 1 in FIG.
04 (Display (Analog)) indicates the timing at which an analog video signal is displayed.

On the other hand, the video signal after the overlay processing is I,
In order to perform re-encoding with the same parameters as the first encoding including the P and B picture types, it is necessary to perform re-encoding in the same order as the decoding order indicated by reference numeral 100. Therefore, it is possible to start re-encoding from the time when the frame I2 to be decoded first is displayed. Reference numeral 105 (Video Re-encoding) in FIG. 4 indicates the re-encoding timing of the video signal after the overlay processing. The re-encoded moving image data is multiplexed with a similarly re-encoded audio signal and output as digital data.

On the receiving side (not shown), the encoded digital data is decoded after the transmission / reception buffer delay and the transmission path delay, and the frame is rearranged in accordance with the decoding. Will be
Reference numeral 106 (Video Re-decoding) in FIG. 4 indicates the decoding timing on the receiving side, and reference numeral 107
(Display (Digiatal)) indicates the timing at which the message is displayed.

[Second Embodiment] FIG. 5 is a block diagram showing a configuration of a system including a moving picture re-encoding device according to a second embodiment of the present invention. In FIG. 5, the same components as those in the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and differences from the first embodiment will be described.

The difference of this embodiment from the first embodiment is that the output video signal 58 of the video overlay processing unit 18 is different from that of the first embodiment.
And a selector 67 for selecting one of the output video signals 66 from the frame memory 23, the data memory 31 for temporarily storing encoding parameters is removed, and a buffer memory for the sub-video data 53 and the navigation data 54 are provided. 32, 33 are FIFO (First In First O)
ut) The point is that the memory is changed to a RAM (random access memory), and the capacity of the buffer memories 11 and 12 for the audio data 51 and the moving image data 51 is increased.

Further, the difference in operation is that the processing of the overlay processing unit 18 is performed not in the order of frames displayed as in the first embodiment, but in the order of frames in which the moving image data 52 is decoded. The points are different.

(Operation of Moving Picture Re-encoding Apparatus) FIG.
The operation of the present embodiment will be described with reference to FIG.

FIG. 6 is a diagram showing the operation timing of the present embodiment, and corresponds to FIG. 4 showing the operation timing of the first embodiment. In FIG. 6, reference numerals 120, 1
21 and 122 are reference numerals 100 and 10 in FIG.
The timings of decoding moving image data, the timing of displaying moving image data, and the timings of decoding and displaying sub-pictures corresponding to reference numerals 1 and 102 are shown.

In the first embodiment, decoding and display are performed in accordance with the timings 100, 101, and 102 shown in FIG.
Decoding and re-encoding processes are performed at timings 3 to 126, and analog video image signals and digital video data are output.

First, the decoding of each moving picture data is started not by the original DTS but by the PTS of the moving picture frame I2 to be decoded first. In accordance with the delay time of the decoding start time, the moving image data 52 read from the optical disk 50 is accumulated using the capacity added to the buffer memory 12. Audio data 5
1, sub-picture data 53 and navigation data 54 are the same. Reference numeral 123 in FIG. 6 indicates the decoding timing of the moving image data in the present embodiment.

Next, decoding and overlay processing of the sub-picture signal which is originally subjected to the overlay processing in the displayed frame order are performed in accordance with the order in which the moving image frames are decoded. Reference numeral 124 in FIG. 6 indicates the timing of decoding the sub-video signal and the overlay processing on the decoded moving image frame. Since the decoding order of the sub-pictures is different from the original decoding order, the sub-picture data 53 and the navigation data 54 are read from the buffer memories 32 and 33 by random access to necessary data, respectively.

An analog video output 68 sent to the analog signal system 62 is switched by a selector 67 between a video signal 58 that has been subjected to overlay processing and a video signal 66 that has been input to the encoder 22 and recorded in the frame memory 23, and is output. The selector 67 selects the video signal 58 when the moving picture frame to be output is a B picture, and selects the video signal 66 read from the frame memory 23 when the moving picture frame to be output is an I picture or a P picture. This makes it possible to arrange the moving image signals 58 output in the decoding order in the display order and output the analog signals. Reference numeral 125 in FIG.
Shows the output timing of the analog video signal.

On the other hand, the video signal 58 output from the overlay processing unit 18 in the decoding order is input to the re-encoding unit 34 and re-encoded in the input order, that is, the decoding order. The encoder 22 for re-encoding has the same basic configuration as that of the first embodiment, and uses an encoding parameter 64 extracted from the first encoded data and a predetermined motion vector 65 given from the outside. It is coded adaptively for each macroblock. In the present embodiment, since re-encoding is performed simultaneously with decoding, the data memory 31 for temporarily storing the encoding parameters in FIG. 1 is not required.
Further, since the frame rearrangement for re-encoding is unnecessary, the frame memory for the frame rearrangement required in the first embodiment is not required, and the delay amount from the start of the decoding is reduced by the first. It becomes possible to make it smaller than the embodiment. Reference numerals 127 and 128 in FIG. 6 indicate timings at which the data re-encoded on the receiving side is input, decoded, and displayed.

(Regarding Frame / Field Adaptive DTC and DCT Type) FIG. 7 is a diagram for explaining a DCT (discrete cosine transform) type used in MPEG2 or the like as a typical example of orthogonal transform. In MPEG2, a macro block composed of 16 pixels × 16 lines in a frame is divided into four luminance blocks composed of 8 pixels × 8 lines and two chrominance blocks, and each is subjected to DCT processing. Here, whether the luminance block is a block composed of frame lines or a block composed of lines of each field can be changed for each macro block.

Changing the luminance block to a block composed of frame lines or a block composed of lines of each field for each macroblock is called frame / field adaptive DCT. The information indicating whether or not the block is a luminance block is called a DCT type. FIG. 7A shows a macro block having a frame configuration, FIG. 7B shows division into frame blocks, and FIG. 7C shows division into field blocks.

In the first and second embodiments according to the present invention described above, the macroblock re-encoded by using the motion vector extracted from the first encoded data has the above-mentioned DCT type. Also uses a DCT type extracted from the first encoded data, and adaptively selects which DCT type to use for a predetermined motion vector or a macroblock for which intra-coding has been selected. ing. These DCTs
The type is determined by the motion compensation mode and D in FIG.
This is performed by the CT type determination unit 72. The DCT type adaptive selection calculates a correlation between lines in adjacent frames and a correlation between adjacent lines in the same field. If the intra-frame line correlation is large, the DCT of the frame block is calculated. Is larger, the DCT of each field block is selected.

(How to give a predetermined motion vector)
Next, a method of specifying the default motion vector 65 in FIGS. 1, 2 and 5 will be described. In the first method of specifying a default motion vector, a vector having a size of zero is given as the default motion vector. Sub-images such as subtitles, GUI images, and the like, which are overlaid on the decoded video signal, are generally displayed stationary for a certain period of time. In the area subjected to such overlay processing, coding efficiency is improved by performing predictive coding using a motion vector having a motion amount of zero as compared with the motion vector extracted from the first coded data. . Further, in a frame in which the display or non-display of the overlay image is switched, the prediction efficiency between frames generally decreases, and in such a portion, intra-coding may be performed instead of predictive coding.

Therefore, by selecting the optimum one for each macroblock from among the motion compensated prediction coding based on the motion vector extracted from the first coded data, the prediction coding based on the zero vector, and the intra coding. Even when the overlay processing is performed, the optimum encoding mode is automatically selected.

In the second method of specifying a predetermined motion vector, the predetermined motion vector is determined from the display position and the amount of motion of the image to be overlaid. The former indicates where in the screen the sub-video or GUI image has been overlaid, and the latter indicates the amount of motion when the overlaid image temporally moves.

FIG. 8 shows an example of a temporal change in the overlay processing. In FIG. 8, (a) → (b) →
It changes in the order of (c). In FIG. 8B, the overlay display is started, and the time 110 and the caption 111 are displayed.
The subtitle 111 scrolls in the horizontal direction, and moves to the left as shown by the subtitles 113 and 114 in the frame of FIG. The display positions at times 110 and 112 are fixed, and in coding a macroblock corresponding to these positions, the default motion vector 65 is set to a zero vector. For the subtitles 111, 113, and 114 to be scrolled, the scroll amount is given as a predetermined motion vector. By doing so, it is possible to improve the prediction efficiency at the time of re-encoding, and also to improve the encoding efficiency, for an overlay image that involves temporal translation.

(Second Motion Compensation Mode Judgment Method) Next, the motion compensation mode and the second motion compensation mode judgment method in the DCT type judgment section 72 in FIG. 2 will be described. The motion mode is determined by determining the motion vector 64 extracted from the first encoded data and the predetermined motion vector 6.
5, a motion vector having higher prediction efficiency is selected. In the above description, a method using a vector having a smaller prediction error is used. In this case, reading of the encoding target macroblock image, reading of the reference macroblock image corresponding to each motion vector, and each prediction error Must be calculated.

On the other hand, in the second motion compensation mode determination method, for each macroblock, which motion vector is used is selected from the area ratio of the overlay image occupying the macroblock and the mixing ratio with the video signal.
This makes it possible to reduce the amount of computation for determining the encoding mode.

FIG. 9 is a diagram for explaining an example of coding mode determination. In FIG. 9, reference numeral 111 denotes a portion to be subjected to overlay processing, and reference numerals 130 and 131 show enlarged portions of a macroblock at the time of re-encoding. Since most of the pixels of the macro block 131 are subjected to the overlay processing, the original video signal is almost overwritten. In such a macroblock, prediction efficiency can be improved by using a predetermined motion vector according to an image to be overlaid rather than a motion vector extracted from the first encoded data.

On the other hand, as shown in the macro block 130, when the ratio of the overlay area in the macro block is low, the coding efficiency is maintained by using the motion vector extracted from the first encoded data. It becomes possible. Also, for a macroblock on which no overlay processing is performed, the motion vector extracted from the first encoded data is used.

Further, when the overlay processing is not a pixel overwriting but a mixing processing with a decoded video signal, the motion is calculated by using the product of the area of the macroblock to be overlaid and the mixing ratio. Determine the choice of vector.

(Regarding Video Encoder 22 Having Motion Vector Scaling Function) FIG. 10 is a diagram showing a second configuration example of the video encoder 22 in FIGS. 1 and 5. 10, the same reference numerals as those in FIG. 2 indicate the same components as those in FIG.

The first of the video encoder 22 shown in FIG.
The difference from the configuration example is that a motion vector scaling unit 77 for the motion vectors 64 and 65 is added. The video encoder shown in FIG. 10 converts the whole or a part of the video signal to resolution conversion, that is, enlarges or reduces the video signal, and outputs the frame frequency (frame rate) for decoding and overlay processing of moving image data by the decoding unit. ) Or when the field frequency (field rate) is converted and output.

That is, when re-encoding a moving image signal to which a resolution conversion for enlargement or reduction has been performed, the motion vectors 64 and 65 are also adjusted by the motion vector scaling unit 77 to the resolution conversion ratio (enlargement or reduction ratio) of the video. It is used after being scaled correspondingly.

FIG. 11 shows an example in which an enlarged display image in which the lower left portion of the decoded and overlaid video frame 140 is doubled both horizontally and vertically is output. The macroblock 142 in the video frame 140 will be re-encoded as four macroblocks 143 in the enlarged video frame. In this case, the configuration is such that the value of the motion vector corresponding to the macroblock 142 is scaled twice in both the horizontal and vertical directions, and the scaled motion vector is re-encoded adaptively to each of the four macroblocks 143.

When re-encoding a moving image signal having undergone the conversion of the display frame rate or the display field rate, the input motion vectors 64 and 65 are also converted by the motion vector scaling section 77 into the display frame rate or the display field rate. It is used after being scaled according to the rate conversion ratio.

That is, when the decoded video signal shown in FIG. 12A is thinned out frame by frame and output as shown in FIG. 12B, the decoded video signal is detected from the first encoded data. The video signal shown in FIG. 22B is re-encoded by using the motion vector 202 obtained by scaling 201 of the motion vectors 200 and 201 both horizontally and vertically.

(Regarding Special Playback) Next, in the first and second embodiments according to the present invention, fast forward,
The operation in the case of having a special playback function such as reverse playback, pause, and slow playback will be described.

FIG. 13 shows an example of fast-forward playback, in which only an I-picture portion is extracted from coded moving picture data and decoded intermittently. To output the video of the decoded frame. In FIG. 13, an I picture is encoded at an interval of 15 frames, and decoding is performed at a cycle of 5 frames, thereby realizing fast-forward reproduction at triple speed. In the hatched output frame in FIG. 13, the video of the I picture decoded immediately before is repeatedly output. In this case, upon re-encoding, the hatched frame can be re-encoded as a P picture instead of an I picture. Generally, the first encoded data does not have motion vector information because it is an I picture. However, since the prediction error becomes zero by the motion compensation using the zero vector, the motion compensation mode of the zero vector is automatically selected, and the encoding efficiency at the time of re-encoding becomes very high. It is possible to reduce the transmission bit rate after re-encoding.

FIG. 14 shows an example of reverse reproduction at 3 × speed.
3, only the I picture is decoded, and the order of the frames to be decoded is temporally reversed.
Reverse playback is performed.

FIG. 15 shows an example of the pause operation. In the example of FIG. 15, after the frame I2 is decoded, the same frame is repeatedly displayed to enter the pause state. Also in this case, at the time of re-encoding, similarly to FIG. 13, the hatched frame is re-encoded as a P picture, and predictive encoding using a zero vector is automatically selected.

FIG. 16 shows another method of realizing fast forward reproduction. In FIG. 16, coded data of only an I picture and a P picture is decoded and output, and a frame indicated by hatching in the figure repeatedly outputs the video of the immediately preceding frame. Therefore, FIG.
In the example of, a fast forward reproduction at 1.5 times speed can be realized. Here, the hatched frame is shown in FIG.
And re-encode it as a P picture using a zero vector. In addition, the frame not shaded is re-encoded by using the motion vector and the encoding parameter extracted from the first encoded data, so that the frame is re-encoded without lowering the encoding efficiency. It is possible.

FIG. 17 shows an example of realizing slow reproduction. In the example of FIG. 17, decoding of all encoded video data is performed at a timing that is an integral multiple of the frame period,
When decoding is not performed, the same display frame is repeatedly displayed. The hatched frame in FIG. 17 is a frame that is repeatedly displayed. Here, decoding is performed every three frames, thereby realizing slow playback at 1/3 speed.
The signal indicated by Output 1 in FIG. 17 is an analog output video signal. On the other hand, the re-encoded output at the time of slow reproduction is represented by Out in FIG. 17 in consideration of the structure of inter-frame prediction.
The frame configuration is indicated by put2.

Here, the frames B0 and B1 are obtained by repeatedly re-encoding the same moving image frame as a B picture, and P2 * is the same moving image frame as I2.
Since the reference image and the predicted image match at this time because the reference image and the predicted image match, the prediction error signal is always zero by performing the motion compensation prediction coding of the zero vector, and the coding efficiency is very high. Is performed. Out
The moving image data re-encoded as output 2 is rearranged on the receiving side according to a normal decoding procedure, and is reproduced as a 1 / 3-speed video similar to that of Output 1.

At the time of re-encoding shown in FIGS. 13 to 17, motion-compensated predictive encoding is performed using a motion vector extracted from the first encoded data or a zero vector. Header data indicating the display order of encoded frames (temporal_referenc in MPEG2)
e).

In MPEG2, there is a rate converter for converting a coding frame rate called a 3: 2 pulldown from a coding frame rate to a display frame rate, and each coding frame is displayed for a 3-field period or a 2-field period. (Repeated in MPEG2)
at_first_field) and a flag indicating whether each frame is displayed from the top field or from the bottom field (in MPEG2, top
_Feld_first). In displaying the video signal after decoding, these flags need to be set correctly so that the top field and the bottom field are always displayed alternately. In the examples of FIGS. 13 to 17, the decoded frame is further repeatedly displayed, so that the flag of the header data is also repeated so that the top field and the bottom field are always displayed alternately in the display. Is rewritten after being appropriately corrected according to the display structure.

In the above trick play, when the encoded video data is interlaced video, the decoded video signal is output as it is as shown in FIGS. In the playback of a repeated frame, a jerky moving image may occur. This is because, in an interlaced frame, the second field of the same frame is a video that has elapsed by one field time from the first field, and when the first field is displayed again, the displayed video becomes the past video. This is because it returns to the video and the time goes back and forth. Normally, in order to prevent such unnatural motion, one of the first and second fields is selected in the same frame, and the same field is used in the first and second fields. A method of outputting an image is used. In this case, each macroblock of the first encoded data does not always coincide with each macroblock at the time of re-encoding.

As described above, when a frame is formed by repeating the same field during special reproduction of an interlaced image, re-encoding is performed using the method shown in FIG. In FIG. 18, 0, 1, 2, and 3 indicate field numbers, and (a), (b) and (c), (d) indicate the first number.
2 shows continuous encoded frames in the encoded data of FIG. (A '), (b') and (c '),
(D) shows a configuration at the time of re-encoding a video frame obtained by decoding the encoded data. The frames in FIGS. 18A and 18B are interlaced frames.
In this case, the frames (a ') and (b') to be re-encoded comprise a field in which 0 and 2, which are the top fields of (a) and (b), are displayed repeatedly.

That is, the decoded video fields are in the order of fields 0 → 1 → 2 → 3, but the actual analog display and recoded fields are 0 → 0 →
It is configured in the order of 2 → 2. At this time, if the first encoded data is motion-compensated and predicted from field 0 in field 2 in FIG. 18A, the second encoding is performed using the motion vector 220 in the first encoding. Is performed in units of frame macroblocks. That is, the motion vector 223 used at the time of re-encoding in FIG. 18 is obtained by converting a motion vector in units of 220 fields into a motion vector in units of frames. In the first encoding, when the field 2 is encoded by motion compensation prediction from the field 1, that is, when encoding is performed using the motion vector 221 in FIG. Is vertically and horizontally doubled to generate a motion vector 223 for re-encoding. Thus, the motion vector 2 indicating the motion amount at the time of one field interval
21 is a motion vector 22 indicating a motion amount at one frame interval.
3 can be converted.

As shown in (c) and (d) of FIG. 18, when the first encoded data is a frame image (progressive image) in which interlace is not performed and motion compensation is performed by a frame macroblock. , The motion vector 22 of the first encoded data also in the re-encoding.
2 is used as it is as the motion vector 224 at the time of re-encoding, and the field configuration is 0, 0 of the first encoded data.
Re-encoding is performed in the same manner as in 1, 2, and 3.

Further, as in each of the above-mentioned special reproduction processes,
When a frame is formed by repeating the same field in order to prevent jerky movement, the vertical correlation in the frame is high. Therefore, as the DCT type, the DCT in which the frame block is always a luminance block is selected, so that the coding efficiency is improved. Can be improved.

[Third Embodiment] FIG. 19 is a diagram showing a system configuration of a moving picture re-encoding device according to a third embodiment of the present invention. In the figure, the components denoted by the same reference numerals as those in FIGS. 1 and 2 are the same as the components in FIGS. 1 and 2, and the description thereof will be omitted.

The characteristic difference between the present embodiment and the first and second embodiments is that the re-encoding unit 28 receives the first encoded data 150 and performs the conversion processing on the first encoded data 150.
51 and a selector 152 for selecting the encoded data output from the video encoder 22 and the first encoded data after the conversion processing from the conversion processing unit 151 are added. The output of the selector 152 is re-encoded. The point is that it is configured to be output as the obtained moving image data.

That is, in the present embodiment, when the video signal obtained by decoding the first encoded data is output as an analog signal without any processing, the encoded data obtained by re-encoding this video signal Rather, the corresponding first encoded data is output as it is, and if the video signal obtained by decoding is subjected to overlay processing or special reproduction, the video signal after this processing is processed. In this configuration, encoded data obtained by re-encoding a signal is selected and output.

The switching of the encoded data to be output is performed in units of frames or macroblocks. In the switching in units of macroblocks, if the target macroblock to be coded and the reference image area for motion compensation prediction of the macroblock are not affected by the overlay processing, the corresponding first coded data is output as it is. In the case of, the moving image data obtained by the re-encoding is selected.

Further, in the special reproduction processing shown in FIGS. 13 to 17, if a frame is not generated by repeating the same field, the first encoded data is not replaced with the re-encoded moving image data. The data conversion processing unit 151 rewrites the data in the header area related to the display order and the field configuration described above, and outputs it.

FIG. 20 is a flowchart showing the operation of the selector 152 in FIG. 19 in this embodiment. That is, in the case of special reproduction (trick reproduction) (Yes in step S1), a non-interlaced frame, that is, a progressive frame (Yes in step S2), an overlay or the like is applied to the decoded video signal. Is not performed (step S
3), or if the video signal decoded by normal reproduction has not been subjected to pixel processing such as overlay, the first encoded data (stream processing output) is selected and output (step S4). In other cases (No in step S2 or No in step S3), re-encoded data is selected and output (step S5).

(Regarding Motion Vector Scaling According to Interframe Prediction Structure) FIG. 21 shows an example of motion vector scaling according to the interframe prediction structure at the time of recoding according to the embodiment of the present invention. FIG. 21A shows the first
, And I0, B1, B2, P3,... Represent the I picture, B picture, and P picture of the frame structure in the order of display. FIG. 21B illustrates an inter-frame prediction structure at the time of re-encoding using one picture having a frame structure and P pictures having a continuous frame structure. FIG. 21C shows an inter-field prediction structure for re-encoding using one picture having a field structure and P pictures having a continuous field structure. 500 ~
The arrow indicated by 521 indicates a motion vector of the first encoding and re-encoding, and the starting point of the arrow is an encoding target image, and the ending point of the arrow is a reference image.

In the present embodiment, the first coded data obtained by coding a moving picture signal with a combination of one picture, P picture and B picture having the frame structure shown in FIG. By re-encoding the moving image signal obtained by this decoding using the prediction structure shown in FIG. 21B or 21C, it is not necessary to rearrange frames at the time of re-encoding. Therefore, a frame memory for rearranging frames in re-encoding is not required, and the delay of re-encoding can be reduced.

Further, in the example of FIG. 21 (c), since encoding is performed using a picture having a field structure, it is possible to further reduce a field memory for field / frame conversion and a delay amount at the time of re-encoding. It becomes possible.

However, in FIGS. 21B and 21C, since the inter-frame prediction structure is different from that of the first encoded data,
Motion vectors 500 to 5 extracted from the encoded data of
05 cannot be used at the time of re-encoding as it is.
However, by scaling the motion vector extracted from the first encoded data in consideration of the difference in the prediction structure, the motion vector can be used at the time of re-encoding.

In FIG. 21B, P1, P2, P3
Are frames B1, B2, P3 in FIG.
And the motion vector 500 of the first encoded data can be used as it is as the motion vector 510. The motion vectors 511 and 512 are the motion vectors 501 and 502 of the first encoded data.
Can be used which are scaled by 1/2 times and 1/3 times, respectively.

Also, in the predictive coding of the field structure as shown in FIG. 21C, in the case of MPEG2 coding, either the immediately preceding in-phase field or the opposite-phase field can be used as a reference image. Here, i0 and p1 correspond to the fields constituting the frame of I0 in FIG. 21A, and p2 and p3 correspond to the fields constituting B1 in FIG. 21B. Therefore, the motion vector 500 of the first encoded data can be used as the motion vector 520, and the motion vector 52
1 is the motion vector 500 of the first encoded data.
Can be used that is scaled by a factor of two.

That is, in order to reduce the cost and delay time of the re-encoding unit, even if the inter-frame prediction structure at the time of re-encoding is different from the first coded data, it is extracted from the first coded data. Scaled motion vectors in consideration of the difference in prediction structure, enabling re-encoding by predictive coding using motion-compensated prediction without re-detecting the motion vector during re-encoding. Becomes

(Regarding Display Switching of Sub-Video Signal or GUI Image) Next, a method of switching display of the sub-video signal or GUI image according to the embodiment of the present invention will be described. As a method of displaying a movie with a frame rate of 24 Hz as an interlaced image of a standard television with a frame rate of 29.97 Hz, there is a method called 3: 2 pull-down. The 3: 2 pulldown allows a frame of a movie to be displayed at a frame rate of 29.97 Hz by alternately switching between displaying one frame of a movie for each frame for a three-field period or displaying a two-field period. In the display in the three-field period, the video displayed in the third field is a repetition of the first field in the same frame. In MPEG2 encoding,
Encoding corresponding to 3: 2 pulldown is possible. That is, in encoding a movie or the like, encoding is performed in units of a frame image having a frame rate of 24 Hz, and whether the encoded frame is a frame displayed for a 3-field period or a frame displayed for a 2-field period is encoded. Indicated by a flag in the data, the encoding frame rate is changed from 24 Hz to the display frame rate in the decoding device.
Converted to 97 Hz.

FIG. 22 is a diagram showing an example of switching the display of a sub video signal or a GUI image according to the embodiment of the present invention. FIG. 22A shows display field numbers.
(B) shows the configuration of an encoded frame corresponding to 3: 2 pull-down processing. In the example of FIG. 22, the frame composed of fields 0 and 1 is the first encoded frame, and the frame composed of fields 2, 3 and 4 is 2
This is the third encoded frame. Here, field 4 is deleted at the time of encoding, and after decoding, field 2 is repeatedly displayed at the timing of field 4.
A field indicated by a dotted line in FIG. 22B is a field repeated by 3: 2 pull-down, and is shown in FIG.
(C) shows the field configuration after decoding.

FIG. 22D shows the sub video signal or the GUI.
In this example, the first sub-video or GUI image is displayed statically in fields 0 to 7 and the second sub-video or GUI image is displayed in fields 8 to 16 in this example. Is displayed statically. Here, the switching of the sub-video signal or the GUI image signal refers to the on / off state of the display itself or the mixing ratio when the sub-video or the GUI image is faded by gradually changing the mixing ratio with the video signal. And the like.

In the example of FIG. 22D, at the time of field 7, the first sub-picture or GUI image is overlaid, and at the time of field 9, the second sub-picture or GUI image is overlaid. . Therefore, the video of field 9 where the same video as the video of field 7 is originally displayed does not match the video of the time of field 7 after the overlay processing.

When the video signal after the overlay processing is re-encoded with the same frame configuration as the first encoded data, that is, the video at the time of field 9 is
When the encoded data obtained by the re-encoding is decoded, the displayed sub-image or GUI image is displayed in the order of FIG. As a result, the image is disturbed with an unnatural time change.

On the other hand, as shown in FIG. 22 (f) or (g), the changing point of the sub-picture signal or the GUL image is shown in FIG.
If the video signal subjected to the overlay processing is re-encoded with the same frame configuration as the first encoded data by shifting to the boundary between the fields constituting the encoded frame shown in (a), the encoded image is re-encoded. When displaying the video signal obtained by decoding the data, the sub video signal or the GU1 image is not temporally disturbed, and a smooth video can be displayed.

[0124]

As described above, according to the present invention, at the time of decoding coded moving picture data, a coding parameter including a motion vector, a coding mode, and a quantization step is extracted, and the decoded base data is extracted. The band video signal is subjected to special playback processing such as fast-forward, slow, and pause, and processing such as sub-video or GUI overlay. For unprocessed frames or regions, the extracted coding parameters are used. Re-encode, or modify and output a part of the original coded data, and convert and generate motion vectors for areas where special reproduction and overlay processing have been performed according to the contents of each processing. And re-encoding, it is possible to realize high-quality and low-cost re-encoding with a high degree of freedom in special reproduction and overlay processing.

Therefore, with the moving picture re-encoding device of the present invention, digital connection between digital video systems of different systems can be smoothly performed.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a system including a moving image re-encoding device according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration example of a video encoder for re-encoding in the embodiment.

FIG. 3 is a diagram illustrating an example of a graphics overlay image.

FIG. 4 is a diagram showing an example of operation timing of the moving picture re-encoding device according to the embodiment;

FIG. 5 is a block diagram showing a configuration of a system including a moving picture re-encoding device according to a second embodiment of the present invention.

FIG. 6 is an exemplary view showing an operation timing example of the moving picture re-encoding device according to the embodiment;

FIG. 7 is a view for explaining a DCT type regarding MPEG.

FIG. 8 is a diagram illustrating an example of a graphics overlay image.

FIG. 9 is a diagram illustrating an example of a graphics overlay image.

FIG. 10 is a block diagram showing another configuration example of a video encoder for re-encoding in the present invention.

FIG. 11 illustrates an example of a graphics overlay image.

FIG. 12 is a diagram illustrating an example of motion vector scaling accompanying frame rate conversion.

FIG. 13 is a diagram showing an example of fast-forward playback according to the present invention.

FIG. 14 is a diagram showing an example of fast-forward reverse playback according to the present invention.

FIG. 15 is a diagram showing an example of pause reproduction according to the present invention.

FIG. 16 is a diagram showing an example of fast-forward playback according to the present invention.

FIG. 17 is a diagram showing an example of slow playback according to the present invention.

FIG. 18 is a diagram showing an example of motion vector scaling according to the present invention.

FIG. 19 is a block diagram showing a configuration of a system including a moving image re-encoding device according to a third embodiment of the present invention.

FIG. 20 is a diagram showing a control flowchart according to a second embodiment of the present invention.

FIG. 21 is a diagram showing an example of motion vector scaling according to the embodiment of the present invention.

FIG. 22 shows a sub-video signal or G according to the embodiment of the present invention.
FIG. 9 is a diagram illustrating an example of switching display of a UI image.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Data separation part 11 ... Audio data buffer memory 12 ... Moving image data buffer memory 13 ... Sub-picture data buffer memory 14 ... Navigation data buffer memory 15 ... Audio decoder 16 ... Video decoder 17 ... Sub-picture decoder 18 ... Video overlay processing part Reference Signs List 20 GUI image signal generator 21 Audio encoder 22 Video encoder 23 Frame memory 24 Audio data buffer memory 25 Moving image data buffer memory 26 Multiplexer 27 IEEE 1394 interface 28 Re-encoder 29 Audio Signal D / A converter 30 ... Video signal D / A converter 50 ... Optical disk device 67 ... Video signal selector 70 ... Motion compensation unit 72 ... Motion compensation mode and DCT type determination unit 73 ... Rate control unit 74 ... DC T and quantization unit 75: inverse quantization and inverse DCT unit 76: variable length encoding unit 151: moving image data conversion unit 152: moving image encoded data selector

Continued on the front page F term (reference) 5C053 FA14 FA24 FA27 GB19 GB22 GB29 GB38 KA01 KA25 5C059 KK37 KK38 MA00 MA23 MC14 NN01 NN28 PP04 PP12 RB09 RE07 SS16 TA17 TA31 TA62 TB05 TB07 TC36 UA02 UA05 UA29

Claims (15)

[Claims] A decoding means for decoding first coded data obtained by motion-compensated predictive coding of a moving picture signal to obtain a moving picture signal; and at least a motion vector from the first coded data. Encoding parameter extraction means for extracting an encoding parameter including information on an encoding mode and a quantization step; and a moving image signal obtained by the decoding means, the encoding parameter extracted by the encoding parameter extracting means. And re-encoding means for obtaining second encoded data by re-encoding according to the following: (a) encoding using the same encoding parameter as the extracted encoding parameter ,
(B) motion-compensated predictive coding using a predetermined motion vector; and (c) intra-coding, which is characterized by having a coding mode selecting means for selecting one coding mode for each macroblock. Moving image re-encoding device. 2. A means for generating a sub-video signal or a GUI image signal corresponding to a video frame or a video field of a moving image signal obtained by said decoding means; Means for sequentially performing overlay processing of a video signal or the GUI image signal, wherein the re-encoding means has storage means for temporarily storing the encoding parameters extracted by the encoding parameter extracting means, 2. The moving picture re-encoding device according to claim 1, wherein the video frame or the video field on which the overlay processing has been performed is re-encoded using the encoding parameter stored in the storage unit. 3. A means for generating a sub-video signal or a GUI image signal corresponding to a decoding order of a video frame or a video field of a moving image signal obtained by said decoding means; Means for sequentially performing overlay processing on the sub-video signal or the GUI image signal, wherein the re-encoding means re-encodes the video frame or video field on which the overlay processing has been performed in accordance with the decoding order. The moving picture re-encoding device according to claim 1, wherein the moving image is encoded. 4. The encoding / parameter extracting unit further extracts a frame / field adaptive orthogonal transform type as the encoding parameter from the first encoded data, and the re-encoding unit includes a frame / field adaptive orthogonal transform. Conversion means, and the encoding mode selection means (a)
For a macroblock in which an encoding mode using the same encoding parameter as the encoding parameter is selected, a frame / field adaptive orthogonal transform type extracted from the first encoded data is selected. For a macroblock for which one of the coding modes of (b) motion compensation prediction coding using a predetermined motion vector and (c) intra coding has been selected by the selection means, an optimal frame / field adaptive orthogonal transform type is selected. 2. An orthogonal transformation is performed by the frame / field adaptive orthogonal transformation means.
The moving picture re-encoding device according to any one of claims 3 to 3. 5. The moving picture re-encoding device according to claim 1, wherein the predetermined motion vector is a zero vector. 6. The moving image according to claim 1, wherein the predetermined motion vector is determined from a display position of the sub-video signal or the GUI image signal and a motion amount thereof. Image re-encoding device. 7. The coding mode selection means includes: a display position of the sub-video signal or the GUI image signal and a motion amount thereof; an area ratio of a moving image signal obtained by the decoding means to each macroblock; The moving picture re-encoding device according to any one of claims 1 to 4, wherein the encoding mode is selected using at least one of the ratios. 8. The resolution conversion means for performing resolution conversion for enlarging or reducing at least a part of the moving image signal obtained by the decoding means, and a conversion ratio of the resolution conversion means. And scaling means for scaling the motion vector extracted from the first encoded data or the predetermined motion vector, wherein the encoding mode selecting means performs the scaling after the scaling is performed. The moving picture re-encoding device according to any one of claims 1 to 4, wherein the encoding mode is selected using a motion vector. 9. The rate conversion means for converting a display frame rate or a display field rate with respect to at least a part of the moving picture signal obtained by the decoding means, Scaling means for scaling the motion vector extracted from the first encoded data or the predetermined motion vector in accordance with a conversion ratio, wherein the encoding mode selecting means performs the scaling. The moving image re-encoding device according to any one of claims 1 to 4, wherein the encoding mode is selected using a subsequent motion vector. 10. The re-encoding means according to a reproduction parameter associated with a special reproduction function including at least one of fast-forward, slow, frame-forward and pause of the decoding means. Motion vector conversion means for converting the motion vector extracted from the encoded data, wherein the encoding mode selection means selects the encoding mode using the motion vector converted by the motion vector conversion means. The moving picture re-encoding device according to any one of claims 1 to 4, wherein: 11. The re-encoding means includes: data extraction means for extracting encoded data in frame or macroblock units from the first encoded data; and for the encoded data extracted by the data extraction means, Header data changing means for changing predetermined header data including at least data relating to the display order and the structure of the display field; and a part of the second encoded data extracted by the data extracting means and the header data changed. The moving picture re-encoding device according to any one of claims 1 to 4, further comprising data replacement means for replacing the header data with coded data changed by the means. 12. The coded data replacement means, based on a motion vector extracted from the first coded data, a display position of the sub-picture signal or the GUI image signal, and a motion amount thereof, 12. The moving picture re-encoding device according to claim 11, wherein the replacement is performed on a macro block basis. 13. The coded data replacement means selects and outputs the second coded data during special reproduction including at least one of fast-forward, slow, frame-forward and pause of an interlaced moving image signal, 12. The coded data extracted by the data extracting means and the header data of which has been changed by the header data changing means is selected and output during normal reproduction and special reproduction of a progressive moving image signal. 10. The moving picture re-encoding device according to claim 1. 14. The re-encoding means performs re-encoding with an inter-frame prediction structure different from the first coded data, and according to the inter-frame prediction structure at the time of the re-encoding, Means for scaling a motion vector extracted from the first encoded data, wherein the encoding mode selecting means selects the encoding mode using the scaled motion vector. The moving picture re-encoding device according to any one of claims 1 to 4, wherein 15. The sub-video signal or the GUI image signal to be subjected to the overlay processing in accordance with a boundary of the decoded video frame or video field corresponding to a coded frame in the first coded data. 4. The moving picture re-encoding device according to claim 2, further comprising means for switching the display of the moving image.