JP3166572B2 - Video / audio compression device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image and sound compression apparatus for compressing and coding moving images and sounds, and packetizing both of them and storing them in a file.

[0002]

2. Description of the Related Art In recent years, with the development of information processing technology, digital processing of moving images, sounds, and the like has been performed.

For example, in the case of audio data, a sampling frequency of 48 kHz, and in the case of a 16-bit stereo signal, 1
The data amount per minute is 11.52 Mbytes. The amount of video data is enormous, such as 24 bits per pixel per minute for video signals, 352 × 240 pixels per frame, and about 435 Mbytes for data at 30 frames per second.
Therefore, recording or transmitting such enormous information as it is has many problems from the viewpoint of the capacity of the storage medium, the processing speed, and the like. For this reason, a band compression technique has been developed and put into practical use in which the redundancy is reduced by focusing on the correlation and human sensory characteristics of the signal, and the amount of information is greatly reduced without reducing sound quality or image quality.

[0004] Band compression methods include methods such as predictive coding and transform coding. With these compression methods, data can be compressed by a fraction to several hundredths. In general, data is stored and stored in a compression-encoded format.

Therefore, there is a method described in Japanese Patent Application Laid-Open No. 61-131944 as a method for compressing moving image sound.

[0006] This captures a moving image and a sound at the same time.
Image compression and audio compression are performed on a frame basis, and the coded image and voice are synthesized, packetized and output to a file. This is effective only for the compression method used. However, MPEG (Moving Pict), which is a video and audio compression method widely used at present, is used.
In the “Ures Expert Group”, the moving picture and the audio adopt different coding schemes, and the above-described scheme cannot be used because of a compression scheme in which correlation between frames is obtained.

[0007] Another method of compressing moving image and sound is disclosed in Japanese Patent Application Laid-Open No. 4-285768.

This is a method in which a moving image is temporarily fetched as a compressed code, the sound is fetched as a compressed code, and the fetched audio data and moving image data are connected in frame units. In this method, the MPE
Encoding is also possible in the G system. However, when a forced termination request is issued by the operator during the compression process, for example, when a forced termination request is issued while capturing an image, the sound up to the position where the request is generated is compression-encoded and captured, and further forced. A normal compressed file cannot be created unless the process of linking the image and the sound up to the position where the end request is generated is performed.

[0009]

Therefore, when a request for forced termination by the operator occurs during the compression process,
For example, when a forced termination request is issued while capturing an image, the audio up to the position where the request was generated is compression-encoded and captured, and furthermore, processing is performed to connect the image and the audio up to the position where the forced termination request is generated. Otherwise, a normal compressed file cannot be created, which requires a considerable amount of time from when a forced termination request is generated to when compression is actually completed.

In view of the above, in the present invention, even when a compulsory termination request is issued by an operator during the compression process, the compression process is completed in a short time, and the compressed data up to the position where the compulsory termination request is issued is transmitted. It is an object of the present invention to provide a video / audio compression apparatus that can output a compressed video / audio file. In addition, the present invention can execute a plurality of applications other than the moving image compression processing in parallel, and can efficiently perform the moving image sound compression processing by controlling the application switching interval according to the output state of the display. It is intended to provide a video and audio compression device.

[0011]

SUMMARY OF THE INVENTION The present invention provides the above object,
Device control means for controlling the operation of a plurality of applications; an application controlled by the device control means; input means for inputting instruction data from a keyboard or a mouse; compression processing state output means to a display or the like; The video and audio compression means comprises an I / F processing means for the apparatus control means, and a processing selection means for determining which of the video compression processing, the audio compression processing and the system processing is to be performed. Moving image compression processing means for performing compression encoding of a moving image; audio compression processing means for performing compression encoding of audio; system processing means for combining the compressed and encoded moving image data and audio data; This is achieved by a moving image / audio compression device including an original file device and a compressed file device.

According to the present invention having the above-described configuration, even when a request for forced termination by the operator occurs on the way,
The compression processing is completed in a short time, and the compressed data up to the position where the forced termination is issued can be output as a compressed file of a normal moving image sound. In addition, multiple applications other than the video / audio compression processing can be executed in parallel, and the video / audio compression processing can be performed efficiently by controlling the application switching interval according to the output state of the display. Become.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a moving picture / audio compression apparatus according to the present invention, which is based on MPEG.

The video / audio compression apparatus according to the present embodiment includes a device control means 1 for controlling the operation of a plurality of applications;
Application 2 controlled by device control means 1
A video / audio compression unit 5; an input unit 3 for inputting instruction data from a keyboard, a mouse or the like; and an output unit 4 for outputting a compression state to a display or the like. I / F processing means 10 for interfacing with video processing, processing selecting means 11 for determining which of the moving image compression processing, audio compression processing and system processing is to be performed, and a moving image for performing compression encoding of a moving image A compression processing unit 12, an audio compression processing unit 13 for compressing and encoding audio, a system processing unit 14 for combining the compressed and encoded moving image data and audio data, a compression source file device 15, And a file device 16.

FIG. 2 is a diagram showing the structure of moving image data which has been compression-encoded in accordance with MPEG, and shows the structure of output data from the moving image compression processing means 12. The upper layer is the video sequence, and the video sequence is from the first sequence header to the last sequence and end code.
A sequence, one or more sequence headers and G
It is composed of an OP (Group of Picture). A GOP is composed of one or more pictures as shown in the lower layer, and one picture indicates one image.

As picture types, there are an I picture which can be decompressed only by that frame, a P picture which is decompressed with reference to the previous frame, and a B picture which is decompressed with reference to the previous and next frames. One GOP includes one or more I pictures and zero or more P pictures and B pictures.

In the case of MPEG, the order in which pictures are reproduced and the order in which they are compressed are switched, so that I-pictures and P-pictures following B-pictures are coded before B-pictures.

For this reason, in the present invention, processing from an I picture or P picture to the next I picture or P picture in the reproduction order is performed as one compression unit.

For example, the reproduction order is I1, B2, B3, P4
In the case of B5, B6, P7 ... (I, P, and B represent picture types, and subsequent numbers represent the order of reproduction), the first I
1, B2, B3, P4 is one compression unit, and the next P4, B5, B6, P7 is one compression unit. Here, P4 is included in the two compression units is B2 · B
This is because the data of the I1 and P4 frames are required for the compression of No. 3, and the data of the P4 and P7 frames are required for the compression of B5 and B6. However, it is not necessary to perform the compression process twice for the picture of P4, and if the compression of P4 is performed when the first compression unit is compressed, it is not necessary to perform the compression of P4 in the next compression unit. Also, the compression unit of the present invention is one compression unit from the I picture or P picture to the next I picture or P picture.
The unit of the GOP may be one compression unit, and in this case, the code amount control of image compression (compression control for matching the compressed data amount to the target data amount) is completed in one compression unit. be able to.

FIG. 3 is a diagram showing the structure of audio data that has been compression-encoded in accordance with MPEG, and shows the structure of output data from the audio compression processing means 13. The upper layer is an audio sequence, and the audio sequence is composed of a plurality of AAUs (audio decoding units). The AAU is composed of an AAU header including information such as a synchronization word, a bit rate, and a sampling frequency, and compressed audio data, as shown in the lower layer.

FIG. 4 is a diagram showing the structure of moving picture audio data which has been compression-encoded according to the MPEG standard. The system processing means 14 outputs data of the moving picture compression processing means 12 and output data of the sound compression processing means 13. Is output after packetized and packed as an input, and the compressed file device 1
6 shows a data configuration stored in the storage unit 6. The video and audio data is composed of a plurality of packs. Each pack is composed of a pack header and one or more packets so as to indicate an intermediate layer.
There is a packet. A video packet is composed of a packet header and video data as shown in the lower layer,
When only the video data in the video packet is extracted and successively formed, a series of video sequences shown in FIG. 3 is formed. An audio packet is composed of a packet header and audio data. If only the audio data in the audio packet is extracted and continued, a series of audio sequences shown in FIG. 4 is composed.

FIG. 5 is a diagram showing the structure of the moving picture compression processing means 12, in which a moving picture compression processing control means 601 for controlling the whole moving picture compression processing and a first compression control means are provided.
Original image reading means 602 for reading original image data for each compression unit, and color signal converting means 603 for converting the original image into a color signal format (YCrCb format) usable in MPEG.
A motion search unit 604 for searching the motion of the image of the previous / next frame and the current frame for each block (8 pixels × 8 pixels space in MPEG) area; / Motion predicting means 605 for calculating the difference between the blocks of the subsequent frame and the current frame, DCT means 606 for performing discrete cosine transform, quantizing means 607 for performing quantization, and VLC for performing highly efficient variable-length coding.
Means 608, a compression code buffer 609 for storing compressed image data after VLC, an inverse quantization means 610 for performing inverse quantization, an IDCT means 611 for performing inverse discrete cosine transform, The motion compensation unit 612 includes a motion compensating unit 612 that calculates a new reference frame by adding pixel values of a region of a block of a frame, and a reference frame unit 613 that stores images of previous / next frames to be referred to.

As shown in FIG. 6, the original image data is read for each compression unit by the original image reading means 602 from the compression source file device, and the color signal conversion means 603 reads the Y image.
The data is converted into CrCb data, and the motion
Compression of one frame is started after searching the motions of the images of the subsequent frame and the current frame for each block area. In the case of compressing an I picture, the DCT unit 606 performs discrete cosine transform on the pixel values of the block area of the current frame, quantizes the quantized unit 607, and
08, the data is highly efficiently compressed into a variable-length Huffman code, and the compressed data is sequentially stored in a compression code buffer 609.

Next, in order to decode the compressed image as a reference frame, the quantized data is dequantized by the inverse quantization means 61.
0 and inversely discrete cosine transformed by the IDCT means 611 to calculate a pixel value.
3 is stored. When compressing a P-picture or a B-picture, the motion prediction means 605 stores the values of the pixels in the block area of the current frame and the reference frame unit 613 referred to by the motion searched by the motion search means 604. The difference is calculated by the pixel value of the block area of the previous frame or the previous / next frame, the difference value is discrete cosine-transformed by the DCT unit 606, quantized by the quantization unit 607, and the variable length is changed by the VLC unit 608. High-efficiency compression is performed on the Huffman code, and the compressed data is sequentially stored in the compression code buffer 609.

Next, when compressing a P picture, the quantized data is inversely quantized by the inverse quantizing means 610 in order to combine the compressed image as a reference frame.
The difference value is calculated by the inverse discrete cosine transform by the means 611, and the difference value is added to the value of the pixel of the block of the previous frame stored in the reference frame unit 613 referred to by the motion prediction means by the motion compensation means 612. And stored in the reference frame unit 613.

The moving image compression processing control means 601 controls the entire series of compression processing described above, and performs control for interrupting and restarting the compression processing for a predetermined time.

FIG. 6 is a diagram showing the structure of the audio compression processing means 13; an audio compression processing control means 701 for controlling the entire audio compression processing; an original sound reading means 702 for reading original sound data from a compression source file device; Original sound data extracting means 703 for extracting 1 AAU of original sound data from the read original sound data, and 32 frequency band mapping processing means 704 for performing frequency band mapping processing in 1 AAU units.
Psychological hearing processing means 705 for performing psychological hearing processing, quantization coding means 706 for performing linear quantization and coding,
It comprises a frame forming means 707 for creating compressed data for 1 AAU by adding additional information to the encoded data, and a compression code buffer 708 for storing the compressed data created by the frame forming means.

In the audio compression, as shown in FIG. 6, original sound data for the same time as the time required for reproducing the moving image compressed by the moving image compressing means by the original sound reading means 702 from the compressed original file is read and the original sound data extracting means 703 is provided. The data of 1 AAU (1152 samples in the case of the MPEG audio layer 2) is cut out from the original sound data read in step (1), and the following processing is performed in units of 1 AAU.

The 32 frequency band mapping processing means 704 divides the input signal into 32 sub-band signals using a sub-band analysis filter, and scales each sub-band signal.
The factors are calculated, the dynamic range is aligned, and the psychological hearing processing means 705 performs a fast Fourier transform on the input signal, and calculates the psychological hearing masking using the result,
Calculate the bit allocation for each subband. The quantization coding means 706 quantizes and codes according to the determined bit allocation, and the frame forming means 707 adds a header and auxiliary information to the quantized and coded subband signal, shapes it into a bit stream, and shapes it into a bit stream.
8 is stored.

The voice compression processing control means 701 controls the entire series of compression processing described above, and performs control of interrupting and restarting the compression processing at a fixed time or at intervals of one AAU processing.

FIG. 8 is a diagram showing the structure of the system processing means 14. The moving image data buffer 801 stores the compressed data taken from the moving image compression processing means 12, and the compressed data taken from the audio compression processing means 13. Data buffer 802 for storing video data and audio data, multiplexing means 804 for packetizing and packing moving image data and audio data, a time code counter 803 for adding time data to packs and packets, and packed video and audio And a video / audio compression code buffer 805 that stores the data of

In the system processing, as shown in FIG. 6, moving image data is received from the moving image compression processing means 12, stored in the moving image data buffer 801, audio data is received from the audio compression processing means 13, and stored in the audio data buffer 802. The data is stored, the packet data of the packet size is cut out from the moving image data by the multiplexing means 804, and a packet header and a pack header are added. Add. Also, data of the packet size is cut out from the audio data, a packet header and a pack header are added, and time data at the time of reproduction is taken from the time code counter 803 and added to the pack header and the packet header. The created packs are sequentially stored in the video / audio compression code buffer 805.

Next, the processing of the moving picture audio compression apparatus will be described with reference to FIG.

First, when the operator instructs the start of compression through the input means 3, the apparatus control means 1 calls the moving picture / audio compression means 5.

When the video / audio compression means 5 is called, I
After passing through the / F processing means 10, the control is transferred to the processing selection means 11. The processing selecting means 11 calls the moving image compression processing means 12 first. The moving image compression processing unit 12 opens the compression source file specified by the operator, reads the original image data from the compression source file device 15 by several frames (one compression unit), and starts the moving image compression work.

One compression unit of an image is a GOP unit or a picture range from the I picture or P picture to the next I picture or P picture.

However, in the present system, if processing is continued for a long time, input processing of other applications, a keyboard, a mouse, and the like, display processing on a display, and the like cannot be performed. Must be returned to Therefore, the moving image compression processing means 1
In step 2, the processing is temporarily interrupted during the compression of one compression unit, and the next time the control is transferred to the moving picture audio compression means 5, the processing selection means 11 is notified that the continuation of the moving picture compression processing is to be performed.

The processing selecting means 11 stores the processing to be performed next, and returns control to the I / F processing means 10. The I / F processing unit 10 acquires the current screen display state from the device control unit 1. If the screen display is focused on the screen of the moving picture / audio compression means, the processing is returned to the apparatus control means 1 and the processing is shifted to the processing selection means 11 so as to immediately continue the next processing. If the screen display is focused on another application, the process returns to the device control means 1 so that the other application can be processed.
When the processing of the other application is completed, the device control means 1 calls the moving image / audio compression means 5 again.

When the video / audio compression means 5 is called again, the control is transferred to the processing selection means 11 through the I / F processing means 10.

The processing selecting means 11 calls the moving picture compression processing means 12 in order to continue the previous processing. The above processing is continuously performed, and when one compression unit has been completed, the processing selection means 11 is notified to perform the next audio processing.
The processing is returned to the / F processing means 10.

Next, when control is transferred to the processing selecting means, the voice compression processing means 13 is called. The audio processing unit 13 reads the audio data for the reproduction time of the number of frames (for one compression unit) processed by the moving image compression processing unit 12 from the compression source file device, and starts the audio compression processing. In the case of audio compression processing as well, in the same way as in the case of moving image compression processing, the processing is returned to the processing selection means 11 in fine compression units, and in the case of the last compression, the processing selection means 11 is notified that the next system processing will be performed The process ends.

At this time, if the unit of audio compression is 1 AAU, it is efficient and the maximum AAU is processed within the time of 1 compression unit of the moving image compression processing. Also, the remaining 1A
Audio data less than AU is compressed the next time control is transferred to the audio compression processing means.

Next, when the processing is shifted to the processing selecting means 11, the system processing means 14 is called.

The system processing means 14 takes in the compression code data of one compression unit compressed by the moving picture compression processing means 12 and the sound compression processing means 13 and packetizes and packs them so that the moving picture and the sound can be synchronized. Write to file after compression.

In this way, the processing of the moving image, sound and system is performed for each compression unit.

FIGS. 8 and 9 show the procedure of the processing described above.

In the video / audio compression apparatus of the present invention, the interval at which control is returned to the apparatus control means 1 is changed depending on whether or not the screen of the video / audio compression means has focus, so that the video / audio compression means is displayed on the screen. When the focus is in focus, the compression work is performed at high speed, and when the focus is out of focus, the process is returned to the device control means 1 frequently, so that other applications can be smoothly executed.

FIG. 8 shows the processing procedure when the display display is focused on the screen of the moving image compression processing means. The interval between the processing from the I / F processing means to the apparatus control means is long. For this reason, the moving image compression processing can be performed at high speed by slowing down the processing of other applications. FIG. 9 shows a processing procedure when the display display is focused on the screen of another application, and smoothes the other application by frequently shifting the processing from the I / F processing means to the apparatus control means. Can be operated.

FIG. 10 shows a processing procedure in the case where a request for forced termination is issued by the operator during the compression processing. Even if the operator instructs the processing to be interrupted during the compression processing, the processing is continued. When the processing for the compression unit is completed, control is performed so that all the processing is completed, and up to the point where the processing is completed, it is possible to leave a normal moving image sound compression code file.

[0051]

As described above, according to the present invention, even if a request for forced termination is made on the way, the compression processing is completed in a short time and the compressed data up to the position where the forced termination is issued is normally processed. It can be output as a simple MPEG file. Further, a plurality of applications other than the moving image compression processing can be executed in parallel, and the moving image sound compression processing can be efficiently performed by controlling the application switching interval according to the output state of the display. .

[Brief description of the drawings]

FIG. 1 is a functional block diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of MPEG moving image data.

FIG. 3 is a diagram showing a configuration of MPEG audio data.

FIG. 4 is a diagram showing a configuration of MPEG moving image audio data.

FIG. 5 is a diagram showing a structure of a moving image compression processing unit.

FIG. 6 is a diagram showing a structure of a voice compression processing unit.

FIG. 7 is a diagram showing a structure of a system processing unit.

FIG. 8 is a diagram showing a processing procedure when the focus is on moving image audio compression.

FIG. 9 is a diagram showing a processing procedure when the focus is on another application.

FIG. 10 is a diagram illustrating a processing procedure when a forced termination request is generated.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Apparatus control means 2 Other application 3 Input means 4 Output means 5 Moving image audio compression means 10 I / F processing means 11 Processing selection means 12 Moving image compression processing means 13 Audio compression processing means 14 System processing means 15 Compression source file device 16 Compressed file device 601 Moving image compression processing control means 602 Original image reading means 603 Color signal conversion means 604 Motion search means 605 Motion prediction means 606 DCT means 607 Quantization means 608 VLC means 609 Compression code buffer 610 Dequantization means 611 IDCT Means 612 Motion compensation means 613 Reference frame part 701 Voice compression processing control means 702 Original sound reading means 703 Original sound data extracting means 704 32 Frequency band mapping processing means of 32 705 Psychological hearing processing means 706 Quantization coding means 70 Frame forming means 708 compression code buffer 801 video data buffer 802 the sound data buffer 803 the time code counter 804 multiplex means 805 video and audio compression code buffer

──────────────────────────────────────────────────続 き Continuation of the front page (51) Int.Cl. ⁷ identification code FI H04N 7/24 H04N 7/13 Z (58) Investigated field (Int.Cl. ⁷ , DB name) G11B 20/10 G10L 19 / 00 H03M 7/30 H04N 5/92 H04N 7/24

Claims (3)

(57) [Claims] 1. A moving picture audio compression apparatus comprising a moving picture compression processing means, an audio compression processing means, and a system processing means, wherein said moving image compression processing means compresses a moving image in units of several frames. The audio compression processing means compresses the audio in the same time unit as the unit of compression of the moving image and the time required for reproduction, and the system processing means performs the moving image data compression-encoded by the moving image compression processing means, by combining the audio data compressed and encoded by the audio compression processing unit, the video and audio compression apparatus for sequentially outputted as a compressed file, further comprising a device control means, the compression of the number of frames
The compression process was interrupted several times during the time of the interruption, the process returns to the device control means, processing of the application program
A moving image / audio compression apparatus characterized by causing a CPU to execute processing of an input / output device. 2. A video / audio compression apparatus comprising video compression processing means, audio compression processing means and system processing means, wherein said video compression processing means compresses a video in units of several frames. The audio compression processing means compresses the audio in the same time unit as the unit of compression of the moving image and the time required for reproduction, and the system processing means performs the moving image data compression-encoded by the moving image compression processing means, A moving image audio compression apparatus for combining audio data compressed and encoded by an audio compression processing unit and sequentially outputting the compressed data as a compressed file, further comprising a device control unit, wherein the compression of the audio compression unit is performed.
The compression process was interrupted several times during the time of the interruption, the process returns to the device control means, processing of the application program
A moving image / audio compression apparatus characterized by causing a CPU to execute processing of an input / output device. 3. A video / audio compression apparatus comprising video compression processing means, audio compression processing means and system processing means, wherein said video compression processing means compresses a video in units of several frames. The audio compression processing means compresses the audio in the same time unit as the unit of compression of the moving image and the time required for reproduction, and the system processing means performs the moving image data compression-encoded by the moving image compression processing means, A video / audio compression apparatus for combining audio data compression-encoded by an audio compression processing unit and sequentially outputting the compressed data as a compressed file, further comprising a device control unit, wherein the compression is performed during the system processing.
The process is interrupted several times, and when the process is interrupted , the process is returned to the device control means to process the application program and
A moving image and audio compression apparatus characterized by causing a CPU to execute processing of a power unit .