JPH1175197A - Image signal compressor and method and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize 2-path encoding in real time processing without causing superimposition deterioration by multi-stage coding by generating two kinds of bit streams in the real time processing. SOLUTION: A compressor 3 applies compression processing like the moving picture experts group MPEG method in combination of motion compensation processing and redundancy reduction processing by orthogonal transform such as discrete cosine transform to image data from a pre-processing unit 2. The compressor 3 is roughly divided into a motion vector detector 4 that detects a motion vector from image data supplied from the pre-processing unit 2 and a coder 5 applying coding processing to the image data. The motion vector detector 4 detects a motion vector of each frame for a processing reference time for each frame of a moving image series. The coder 5 obtains coded data of each frame for a time of 1/2 of a processing reference time per each frame of the moving image series.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal compression apparatus and method for generating a bit stream by performing a compression encoding process on a moving image sequence, and records the procedure of such an image signal compression process. It relates to a recording medium.

[0002]

2. Description of the Related Art In recent years, motion-compensation processing (MC) and redundancy reduction processing by orthogonal transform such as discrete cosine transfer (DCT) have been combined with compression coding of a moving image sequence. MPEG (mo
(ving picture experts group) 1 and MPEG2 have come to be widely used.

[0003] Uncompressed video data is converted into an intra-frame coded image (I picture), an inter-frame forward predictive coded image (P picture), and a bidirectional predictive coded image (B picture) by the above-described MPEG or the like. Compressed into such coded images as magneto-optical disks (MO disks;
disc) or the like, or transmitted using a communication line. In this case, it is necessary to keep the bit rate of the compressed video data after the compression encoding equal to or less than the recording capacity of the storage medium or equal to or less than the transmission capacity of the communication line while maintaining the quality of the decompressed video at a high level.

[0004] However, conventionally, one system of coded bit stream is generated at a certain bit rate for an input moving image sequence, and the coded bit stream is recorded on some kind of storage medium or transmitted as it is on a line. Or, it was later read from a storage medium and sent over a line.

[0005]

By the way, according to this method, it is possible to record at the same rate on a storage medium if the line capacity has a sufficient margin for the bit rate. However, when transmitting using a communication line, it is necessary to reduce the bit rate in accordance with the line capacity. In such a case, a new bit stream is obtained by decoding the bit stream data once and converting it back to moving image data at a desired bit rate again.

That is, this method has a weak point in the delay time and the processing amount in a system in which real-time encoding processing is emphasized. That is, two processing times are required in the process of re-encoding. In addition, as a result of such multi-stage encoding, fatal noise due to compression and decompression may be noticeable depending on the required bit rate. In other words, it is inevitable that noise in the second encoding is superimposed on noise in the first encoding.

[0007] In addition, in a single encoding process, it is not possible to allocate an encoding rate in accordance with the characteristics of the arrangement of a moving image sequence. Conventionally, for the entire input image sequence, the amount of generated information after encoding is examined once,
Although there is a method of calculating the second coding rate, this method cannot be used particularly in applications such as recording on a rewritable storage medium where real-time coding processing is important.

That is, in one real-time encoding process, it is required to generate a bit stream for a storage medium and a line transmission, and to assign an encoding rate according to a sequence of an input moving image. .

[0009] The present invention has been made in view of the above circumstances, and can generate at least two types of bit streams by real-time processing, and does not cause superimposition deterioration due to multi-stage encoding. It is an object of the present invention to provide an image signal compression apparatus and method capable of realizing two-pass encoding by real-time processing.

Further, the present invention has been made in view of the above-mentioned circumstances, and it is possible to generate at least two types of bit streams by real-time processing, and without causing superimposition deterioration due to multi-stage encoding. It is another object of the present invention to provide a recording medium which records, as a software program, image signal compression processing capable of realizing two-pass encoding by real-time processing.

[0011]

In order to solve the above-mentioned problems, an image signal compression apparatus according to the present invention achieves real-time processing for one frame of a moving image sequence composed of N frames of image per second. At a given time (1 / N second), a motion vector detecting means for detecting a motion vector for the moving image sequence, and using the motion vector detected by the motion vector detecting means, M Encoding means for encoding (M is an integer of 2 or more) kinds of moving image sequences. Therefore, the encoding process can be performed twice within the reference processing time for one frame.

Further, the image signal compression method according to the present invention is characterized in that:
In order to solve the above-mentioned problem, a time (1 / N second) given to achieve real-time processing for one frame of a moving image sequence composed of N frames of images per second (1 / N second) is used. A motion vector is detected, and M (M is an integer of 2 or more) kinds of moving image sequences are encoded using the motion vector in 1 / N seconds. Therefore, the encoding process can be performed twice within the reference processing time for one frame.

In order to solve the above-mentioned problems, the recording medium according to the present invention has a time required for realizing one-frame real-time processing of a moving image sequence composed of N frames of images per second. (1 / N second), a motion vector detecting step of detecting a motion vector for the moving image sequence, and M (M is 2 or more) in 1 / N second using the motion vector detected in the motion vector detecting step. (Integer) type of moving image sequence.

[0014]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of an image signal compression apparatus and method according to the present invention.

This embodiment is an image signal compression system in which coded data obtained by compressing and coding an input moving image signal is recorded on a recording medium or transmitted through a communication line.

As shown in FIG. 1, the image signal compression system includes an image input device 1 for digitizing an input image and supplying it to a subsequent stage, and a digital image signal from the image input device 1 (hereinafter referred to as image data). ), And a motion compensation process (MC) performed on the image data from the preprocessing device 2.
And discrete cosine transform (DCT)
r) and a compression device 3 for performing a compression process such as a moving picture experts group (MPEG) in combination with a redundancy reduction process by orthogonal transformation.

The compression device 3 is largely divided into a motion vector detection device 4 for detecting a motion vector from image data supplied from the preprocessing device 2 and an encoding device 5 for performing an encoding process on the image data. Here, the motion vector detecting device 4 detects a motion vector of each frame at a processing reference time for each frame of the moving image sequence. On the other hand, the coding apparatus 5 obtains the coded data of each frame in half the processing reference time for each frame of the moving image sequence.

The motion vector detecting device 4 includes a preprocessing device 2
And a motion detector 7 for detecting a motion vector between frames.

The encoding device 5 has a discrete cosine transform (DC
T) unit 8, quantizer 9, variable length encoder 10, inverse quantizer 11, inverse DCT unit 12, adder 13, frame memory 14, motion compensator 15, and addition Table 16
, A switch 17, a buffer memory 18, a buffer memory 19, a quantization controller 20, and a switch 21.

The DCT unit 8 performs discrete cosine transform (DCT) processing on the image data from the adder 16 in units of macroblocks of, for example, 16 pixels × 16 pixels, and converts the data in the time domain into data in the frequency domain. Output to the quantizer 9.

The quantizer 9 quantizes the image data in the frequency domain supplied from the DCT unit 8 with an arbitrary quantization value, and obtains a variable length coding (VLC) unit 10 as quantized data.
And output to the inverse quantizer 11.

The variable-length encoder 10 performs variable-length encoding on the quantized data supplied from the quantizer 9, and switches the compressed image data obtained as a result of the variable-length encoding, that is, the encoded data, to the switch 21. The data is supplied to either the buffer 18 or the buffer 19 via the buffer.

The buffer 18 or 19 buffers the encoded data and supplies it to the output terminal 22 or 23 in the form of a bit stream. The output terminal 22 or 23 supplies the bit stream to a recording medium or a communication line.

The inverse quantizer 11 inversely quantizes the quantized data input from the quantizer 9 and outputs it to the inverse DCT unit 12 as inverse quantized data.

The inverse DCT unit 12 performs an inverse DCT process on the inversely quantized data supplied from the inverse quantizer 11,
Output to the adder 13.

The adder 13 adds the output data of the motion compensator 15 and the output data of the inverse DCT unit 12, and outputs the result to the frame memory 14.

The motion compensator 15 determines whether the motion detector 7 outputs the data from the frame memory 14 to the frame memory 6.
And performs a motion compensation process on the basis of the motion vector calculated for the reference frame via the reference frame, and outputs the result to the adder 13 and the switch 17.

In the encoding device 5 constituted by such units, the bit stream of the image data (encoded data) compressed through the DCT unit 8, the quantizer 9, and the variable length encoder 10 is switched by the switch 21 and the buffer 21. 18 or through the buffer 19.

At the same time, the output of the quantizer 9 is converted into image data through an inverse quantizer 11 and an inverse DCT unit 12, and the image data of the reference frame already reconstructed via a motion compensator 15 and an adder 13 And store in the frame memory 14.

The image data in the frame memory 14 is subjected to motion compensation by the motion compensator 15 using the motion vector obtained by the motion vector detecting device 4, and the reconstructed image data is encoded between frames. In the conversion mode, the adder 16 performs subtraction with the image data input from the preprocessing device 2. That is, the switch 17 is connected to the a side. Also, in the mode of encoding within a frame, the switch 17 is connected to the b side.

The coded data obtained through the variable length encoder 10 is converted into a bit stream through buffers 18 and 19, and is recorded on a recording medium such as a magnetic tape, a magnetic disk, an optical disk, or a wired or wireless medium. Transmit using a line.

In the quantizer 9, the rate processing is controlled by a quantization parameter adjusted by the quantization controller 20 based on the bit allocation processing for forming a bit stream performed in the buffer 18 or 19, and the quantization bit rate is controlled. Is varied. That is, the quantization controller 20 sets the quantization bit rate of the quantizer 9 to the buffer 18 or 1
The control is performed based on the bit allocation process performed in step S9.

At this time, the processing period for the double speed processing is switched by the switch 21. That is, the rate control can be performed in two periods, that is, the period in which the switch 21 is connected to the a side and the period in which the switch 21 is connected to the b side.

In the example shown in FIG. 1, for example, image data obtained from a digital video camera for moving images can be recorded on a storage medium and can be transmitted through a communication medium such as the Internet.

An input image signal for one frame in this embodiment is composed of 704 pixels in the horizontal direction and 480 pixels in the vertical direction. In a macroblock divided into 16 pixels in both the horizontal and vertical directions, 44 pixels in the horizontal direction are used. Pieces, 30 vertically
It is composed of

The input moving image sequence is 3 per second.
0 frame, and the time given to achieve real-time processing for one frame is usually 1/30 second, but the encoding device 5 needs to perform encoding at double speed. 1/6 as the processing time per frame
0 seconds are given.

In this image signal compression system, the motion amount between frames for each macroblock is calculated by the motion vector detecting device 4 as usual at 1 × speed processing, and the result is temporarily stored in a buffer or the like. Then, by processing the encoding device 5 that obtains the actual bit stream at twice the normal speed, the compression / encoding process for viewing can be performed twice.

More specifically, the motion detector 4 calculates the amount of motion for each macroblock for each frame by block matching, and calculates and stores the optimum motion vector value in a normal one-frame sync. . In other words, the processing up to this point can be realized without increasing the circuit scale by using ordinary hardware.

Subsequently, the motion compensator 15 performs motion compensation using the obtained motion vector value to reduce the redundancy of the data in the time axis direction, and the encoding device 5 performs the frequency conversion from the spatial axis direction by the DCT. Weighting is performed on the frequency axis of the data whose redundancy has been reduced by the conversion in the axial direction. At that time, the quantization control by the quantization controller 20 is performed on the two buffers 18 or 19 in a time-division manner.

By performing the above process at twice the general processing speed, the encoding process is performed twice in one frame sync.

Here, it is assumed that a circuit for realizing the compression device 3 of FIG. 1 is constituted by two large-scale integrated circuits (LSI) as shown in FIG. 2 as an example. That is, the motion vector detection device 4 and the coding device 5 of FIG. 1 are made to correspond to the motion detection LSI 24 and the coding LSI 25 of FIG. 2, respectively.

At this time, the motion detection LSI 24 and the encoding L
A digital image signal (image data) is input to both the SI 25. Regarding the clocks that define the operating speed of each LSI, the clock fc having a cycle of half the length of the clock fc1 input to the motion detection LSI 24 is used.
2 is input to the encoding LSI 25. That is, the clock to the encoding LSI 25 in FIG.
The clock fc2 is twice as fast as the clock fc2.

In FIG. 2, a motion vector value MV calculated and calculated at a normal speed using the frame memory 6 by the motion detection LSI 24 is stored in the encoding LSI 25 twice per frame sync for each required reference frame. Are supplied one by one. The encoding LSI 25 outputs the compressed bit streams BS1 and BS2 once for each frame sync.

At this time, the frame sync is represented by a signal FSY as shown in FIG. That is, the period in which the frame sync signal FSY is 0 (Low) and 1 (Hig)
The section h) has the same time, and the combined time of these two sections is one frame sync section (1/30 second). Normally, only one system of bit stream for one frame is generated from one frame of image in one cycle time from when the frame sync FSY becomes 0 to the next zero. Frame Sync FSY
During one period, only the encoding LSI 25 is operated at twice the processing speed to generate two systems of bit streams from one frame of image.

That is, assuming that the input moving image sequence has a temporal arrangement of N0, N1, and N2 as shown in FIG. 4, a motion vector for one frame is provided for each cycle of the frame sync FSY as shown in FIG. MV is motion detection LSI2
In step 4, N0, N1, and N2 are obtained as usual.

Regarding the encoding, first, in the section where the frame sync FSY is 0, one system bit stream B
S1, even in the section where the frame sync FSY is 1, the encoded bit stream BS2 of one system is encoded by the encoding LSI 25.
Output from For example, the coded bit streams BS1 and BS2 of the frame N1 are output in the interval of 0 and 1 in the current frame sync, respectively, using the motion vector value MV of the frame N1 obtained in the period before one frame sync. You.

If this operation is applied, as shown in FIG. 4, for example, a bit stream BS1 having a relatively high bit rate in the first half and a bit stream having a relatively high bit rate in the second half are respectively provided in the first half and the second half of the frame sync. A low bit stream BS2 can be generated. Of course, the reverse bit stream may be arranged.

According to this application example, when a moving image is photographed by a digital camera or the like, a relatively high-rate and relatively beautiful image is recorded on an optical disk or the like, and the communication line having a low transmission path capacity, for example, a communication line used in the Internet, has a smaller image density. Can also transmit low-rate video with priority given to communicable points.

Further, as a different application example, in the first half of the frame sync, the amount of information of the target frame obtained by encoding once is measured and learned, and in the latter half, the encoding of the target frame obtained in the first half is performed. By controlling the weight of the code amount for each frame and macroblock in consideration of the complexity of encoding for each frame and macroblock, more precise encoding becomes possible.

For example, as shown in FIG. 4, first, coding is performed once in the section where the frame sync FSY is 0 for the frame N1 to obtain the code amount generated, and then the frame N1 is set in the section where the frame sync FSY is 1. The encoding is performed again according to the amount of information estimated necessary for the frame N1, and the bit stream at that time is adopted. That is, in the compression process, real-time two-pass processing in which the first encoding result is received and the second encoding result is reflected can be performed.

As described above, in the image signal compression system of the present embodiment provided with the compression device 3 including the motion vector detection device 4 and the encoding device 5, the motion vector detection device 4 has a normal circuit scale and operating state. By using only the encoding device 5 so as to operate at twice the normal speed, it is possible to generate two types of bit streams, for example, a high-rate bit stream and a low-rate bit stream by real-time processing. .

This makes it possible, for example, to record and save a high-rate beautiful image on a storage medium or the like, and to transmit a low-rate image on a network having a low line capacity. At this time, superimposition deterioration due to multi-stage encoding does not occur.

Similarly, two-pass encoding can be performed by real-time processing. This is because deterioration in compression and decompression can be reduced by realizing more precise coding allocation in the latter half from the result of performing the feature extraction of the input video sequence in the first half.

Further, the circuit scale of the present embodiment is accompanied only by an increase for realizing an encoded block corresponding to the double speed processing, and the motion vector detecting block which determines the largest circuit scale is a single speed. It can be realized as it is. Further, it is easy to apply to many applications by changing the variation of the processing.

In the image signal compression system according to the above-described embodiment, the coding device 5 obtains the coded data in the processing reference time for each frame of the moving image sequence, that is, 1/2 time of the frame sync. , 1/3, three types of bit streams can be obtained. It is also possible to obtain more bit streams.

A program for operating the image signal compression system as described above, that is, a motion vector detecting step of detecting a motion vector of each frame at a processing reference time for each frame of a moving image sequence, 1 / of the processing reference time for each frame of the video sequence
If a processing procedure including an encoding step of obtaining encoded data of each frame in the time of 2 is recorded on a recording medium, the above system can be realized by software.

[0057]

According to the image signal compression apparatus and method according to the present invention, at least two types of bit streams can be generated by real-time processing, and superimposition deterioration due to multi-stage encoding does not occur. Further, two-pass encoding can be realized by real-time processing.

The recording medium according to the present invention can generate at least two types of bit streams in real time, and can realize two-pass encoding in real time without causing superimposition deterioration due to multi-stage encoding. Signal compression processing can be provided as software.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an image signal compression apparatus and method according to the present invention.

FIG. 2 is a block diagram in which a motion vector detecting device and an encoding device constituting the above embodiment are integrated.

FIG. 3 is a diagram for explaining a frame sync which is a reference of operation timing in the embodiment.

FIG. 4 is a timing chart for explaining the operation of the embodiment.

[Explanation of symbols]

1 image input device, 2 pre-processing device, 3 compression device, 4
Motion vector detection device, 5 coding device

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Hidehiko Morisada 6-7-35 Kita Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation

Claims (7)

[Claims] An image signal compression apparatus for generating a bit stream by performing a compression encoding process on a moving image sequence, comprising: real-time processing for one frame of a moving image sequence consisting of N frames of image per second. Time to achieve (1
/ N seconds), using a motion vector detected by the motion vector detecting means to detect a motion vector for the moving image sequence, and M (M is an integer of 2 or more) in 1 / N seconds. An image signal compression apparatus, comprising: an encoding unit that encodes a type of moving image sequence. 2. The encoding means generates a bit stream in which the same image is subjected to information amount control of a first encoding rate in 1/2 time of 1 / N second, and generates a bit stream. 2. The image signal compression apparatus according to claim 1, wherein a bit stream in which the information amount of the second encoding rate is controlled is generated in the time period. 3. The encoding means obtains an information amount at a first encoding rate for one half of the 1 / N second for the same image, and obtains the first encoding amount for the remaining time. 2. The image signal compression apparatus according to claim 1, wherein the information amount at the coding rate of (1) is read as the encoding difficulty, and the information amount of the second coding rate is obtained. 4. An image signal compression method for generating a bit stream by performing a compression encoding process on a moving image sequence, comprising: real-time processing for one frame of a moving image sequence composed of N frames of image per second. Time to achieve (1
/ N seconds), a motion vector for the moving image sequence is detected, and M (M
Is an integer of 2 or more) types of moving image sequences. 5. The method of claim 1, wherein 1 / N second is 1 for the same image.
/ 2, generating a bit stream subjected to the information amount control of the first coding rate in the remaining time, and generating a bit stream subjected to the information amount control of the second coding rate in the remaining time. The image signal compression method according to claim 4, wherein 6. The same 1 / N second as 1 / N second for the same image.
/ 2, the amount of information at the first coding rate is obtained, and the remaining amount of information at the first coding rate is read as coding difficulty in the remaining time, and the amount of information at the second coding rate is read. 5. The image signal compression method according to claim 4, wherein 7. A recording medium recording a processing procedure for generating a bit stream by performing a compression encoding process on a moving image sequence, wherein one frame of a moving image sequence composed of N frames of images per second Minutes to achieve real-time processing in minutes (1
/ N seconds) and a motion vector detecting step of detecting a motion vector for the moving image sequence, and using the motion vectors detected in the motion vector detecting step, M (M is an integer of 2 or more) in 1 / N seconds. What is claimed is: 1. A recording medium recording a processing procedure including an encoding step of encoding a moving image sequence of a kind.