JP2002204451A - Code compression/data conversion device and code compression/data conversion method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a code compression/data conversion device and a code compression/data conversion method that enable construction of a system where video audio code compression can easily, automatically and temporarily be stopped when a system load gets heavier without the need for processing by an application program and the video audio code compression can be restarted when the system load gets lighter and that enable immediate resumption of the code compression when the system load gets lighter. SOLUTION: The code compression/data conversion device is provided that applies code compression processing to video and audio or either of them or converts the video and audio data in the system on which programs can work and records or distribute the result in real time and has a function of automatic temporary stop or automatic resumption of the code compression or the data conversion.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image / audio code compression technique, and more particularly to a technique for automatically compressing an image / image / audio code when the load on the system is increased without any processing by an application. A code compression / data conversion apparatus and a code compression apparatus capable of easily suspending and resuming the video / audio code compression when the load of the system is reduced, and capable of restarting the code compression immediately after the load is reduced. And a data conversion method.

[0002]

2. Description of the Related Art Conventionally, an image has been digitized and a CD-R
When recording on a recording medium such as an OM or a hard disk,
Since the amount of data becomes huge, it is usually recorded after code compression. As such a code compression method, there are various code compression methods, and in particular, DCT (DT) which performs compression using the property that the spatial frequency of an image is concentrated at a low frequency.
iscrete Cosine Transform)
Are used relatively frequently. The encoding method is JPEG (Joint Photo).
graphical Expert Group),
MPEG (Moving Pictures Expe
rt Group) 1 or MPEG2.

[0003] A conventional compression operation in the case of MPEG will be described below.

FIG. 4 is a block diagram of the video code conversion control means 10 for performing image compression. In the image compression shown in FIG. 4, a YUV conversion unit 11 converts an original image into a color signal format (YUV format) that can be code-compressed by MPEG, and blocks the movement of the image of the previous / next frame and the current frame (1 in MPEG).
Motion search means 1 for searching each area of (6 pixels × 16 pixels)
2, a DCT unit 13 for transforming the image of the block into a spatial frequency, a quantizing unit 14 for performing quantization, an inverse quantizing unit 17 for performing inverse quantization to return from quantization, and an original to return from frequency transform. Inverse DCT means 16, motion compensation means 15 for compensating the motion of the image in block units to generate a new reference frame, and variable length coding means 1 for performing variable length coding
8.

[0005] In image compression, as shown in FIG.
The data is converted into YUV data by the V conversion means 11, and compression is performed according to each picture type.

In the case of an I picture, YUV data is divided into blocks, converted into spatial frequencies by the DCT means 13 for each block, quantized by the quantizing means 14, and variable-length coded by the variable-length coding means 18 for compression. Output sign. Further, the quantized block is inversely quantized by the inverse quantization means 17 and inverse frequency transformed by the inverse DCT means 16 to create a reference frame.

In the case of a P picture, the YUV data is divided into blocks, and a block having the highest correlation among the blocks of the previous frame stored as a reference frame stored by the motion search means 12 for each block is determined. The difference from the block of the higher previous frame is converted into a spatial frequency by the DCT unit 13, quantized by the quantization unit 14, variable-length coded by the variable-length coding unit 18, and a compressed code is output. Further, the quantized block is dequantized by the inverse quantization means 17.
, And inversely frequency-converted by the inverse DCT means 16 and added to the motion-compensated block by the motion compensation means 15 to create a reference frame.

In the case of a B picture, the YUV data is divided into blocks, and the motion search means 12 finds the block having the highest correlation among the blocks of the previous / next frame stored as the reference frame for each block, and obtains the correlation. Is converted to a spatial frequency by the DCT unit 13, quantized by the quantization unit 14, and subjected to variable-length encoding by the variable-length encoding unit 18 to output a compressed code. In the case of a B picture, there is no need to create a reference frame.

FIG. 5 is a block diagram of the audio code conversion control means 20 for performing audio compression. Audio compression is 1A from the original sound
Original sound extracting means 21 for extracting original sound data for UU
And frequency band mapping means 22 for generating 32 frequency band mappings for performing a frequency band mapping process in units of 1 AAU, quantization coding means 23 for performing linear quantization and coding, and additional information for the coded data. And a frame forming means 24 for creating compressed data for one AAU, and a psychological hearing processing means 25 for performing psychological hearing processing.

In audio compression, as shown in FIG. 5, an original sound is cut out by the original sound cut-out means 21 for one AAU (1152 samples in the case of the MPEG audio layer 2) and compressed in AAU units. The frequency band mapping means 22 decomposes the input signal into 32 sub-band signals using a sub-band analysis filter, calculates a scale factor for each sub-band signal, and adjusts the dynamic range. The psychological hearing processing means 25 performs a fast Fourier transform of the input signal, calculates a psychological hearing mask using the result, and calculates a bit allocation for each subband. The quantization encoding means 23 performs quantization and encoding according to the determined bit allocation. The frame forming means 24 adds a header and auxiliary information to the quantized and encoded sub-band signal, shapes the bit stream into a bit stream, and outputs the bit stream as a compressed code.

FIG. 6 is a diagram showing the structure of the image / audio code conversion control means 30 for multiplexing a video code and an audio code. Referring to FIG. 6, an image / speech code conversion control unit 30 for controlling the entire system coding includes an audio pack generation unit 31 for packing audio codes.
And video pack generating means 3 for packing video codes
2, a padding pack generating means 33 for generating padding packs that are skipped by the decoder in order to adjust the bit rate, and a time stamp generating means 34 for generating a time stamp to be inserted into the packet header.

In the system coding process, as shown in FIG. 6, a video code and an audio code are received by an image / audio code conversion control means 30 and a video pack generation means 32 is provided.
And the audio pack generating means 31 to output the same as a system code.

The video pack generating means 32 cuts out data of a packet from the video code and adds a packet header and a pack header. Further, the audio pack generating means 31 cuts out data for a packet from the audio code and adds a packet header and a pack header.

At this time, in the case of video, if the head of the frame is included in the packet data, the PTS or DTS generated by the time stamp generating means 34 is inserted. In the case of audio, if the head of the AAU is included in the packet data, the PTS generated by the time stamp generating means 34 is inserted. Also,
The padding pack generation means 33 inserts padding packs so that the system bit rate is averaged.

As described above, when the image and the sound are compressed in real time, it is necessary to synchronize the image and the sound by interpolating the smaller code when the difference between the code amounts of the image and the sound becomes large. As such a prior art, for example, there is one described in Japanese Patent Application Laid-Open No. 10-42295 (first prior art).

In the first prior art, a video code is interpolated and synchronized by generating a code indicating that it is the same as the previous and next frames without performing actual compression.

As other conventional techniques, for example, those described in JP-A-10-164556 (second conventional technique) and those described in JP-A-2000-206994 (third conventional technique) are exemplified. is there.

In the second prior art and the third prior art, audio codes are interpolated and synchronized by inserting silent codes.

Still another prior art is disclosed in, for example, JP-A-2000-78531 (fourth prior art).

In the fourth prior art, the audio code is interpolated and synchronized by inserting dummy audio data corresponding to silence.

[0021]

However, the above prior art is based on the premise that either video compression (video code compression) or audio compression (audio code compression) can generate codes continuously. For this reason, if real-time compression is performed by software on a system that can run multiple programs, synchronization of image and audio is performed when a heavy load that other programs run and compression can not be continued is applied And the compression has to be stopped.

When code compression is resumed after the load has been reduced, it is necessary to initialize the capture device and acquire memory and open a file for code compression. There was also a problem that the resumption of gasification was delayed.

The present invention has been made in view of such a problem, and a purpose thereof is to automatically compress image / audio codes when the load on the system increases without processing by an application. A code compression / data conversion device capable of easily and temporarily suspending and resuming the video / audio code compression when the load on the system is reduced, and capable of restarting the code compression immediately after the load is reduced; and It is to provide a code compression / data conversion method.

[0024]

The gist of the invention described in claim 1 of the present invention is to code-compress or convert data of an image and / or sound on a system capable of operating a plurality of programs, A code compression / data conversion device for recording or distributing data in real time, the code compression / data conversion device having an automatic pause or automatic restart function of code compression or data conversion. Further, the gist of the invention described in claim 2 of the present invention is to have means for returning a response to an application without code compression or conversion of image or audio data while code compression or data conversion cannot be continued. The code compression / code compression method according to claim 1,
Exists in the data converter. The gist of the invention described in claim 3 of the present invention is characterized in that there is provided a means for restarting code compression or data conversion at a time such as to reduce the difference in the amount of time between the image code and the audio code. Item 1 is a code compression / data conversion device. The gist of the invention described in claim 4 of the present invention resides in that an interface control that collectively processes code compression or data conversion so that the application does not need to directly participate in code compression or data conversion processing. Means for controlling the application or operating system when the load on the system is increased due to execution of another program and real-time processing cannot be continued. A code compression / data conversion device according to any one of claims 1 to 3. The gist of the invention described in claim 5 of the present invention is that the application operated by the user, the interface control means for controlling a compression operation according to a request from the application, and a video code conversion control for controlling video compression. Means, audio code conversion control means for controlling audio compression, image / audio code conversion control means for controlling system coding,
A video capture buffer for temporarily storing code-compressed or data-converted or captured video data, a video code buffer for temporarily storing generated video codes, and a temporary buffer for code-compressed or data-converted or captured audio data. An audio capture buffer for temporarily storing the generated audio code, an audio code buffer for temporarily storing the generated audio code, and a system buffer for temporarily storing the system code, wherein the interface control means performs code compression or data conversion. Or an original image transfer control unit that passes a captured original image to the video code conversion control unit; an original audio transfer control unit that transfers code compression or data conversion or captured original audio to the audio code conversion control unit; The application has a pause / resume control unit for performing pause and resumption of compression, and the application receives the original image and the original audio that has been subjected to code compression or data conversion or capture, and passes the received original image and audio to the interface control unit. The system code passed to the interface control means is written into a system code file, and the interface control means stores the compressed or data converted or captured original image in the video capture buffer, and performs the code compression or data conversion or capture. Storing the original audio in the audio capture buffer, controlling the video code conversion control means, the audio code conversion control means, and the image / audio code conversion control means, and transmitting the generated code to the video code buffer or the audio codec. Sign Stored in § and the system buffer resides in compacted coded / data conversion apparatus according to claim 4, characterized in that passing the code to the application. The gist of the invention according to claim 6 of the present invention resides in that the application executes a first step of initializing a video and audio capture device, issues a start request to the code compression control, and requires a memory. A third step of opening a file for recording an MPEG code, and a fourth step of determining whether there is code compression or data conversion or captured data from the video capture device. And performing a fifth step of calling the original image transfer control means if there is code compression or data conversion or captured data from the video capture device, and performing code compression or data conversion or If there is no captured data, audio Performing a sixth step of determining whether there is code compression or data conversion or captured data from the audio capture device, and if there is code compression or data conversion or captured data from the audio capture device, passing the original audio Performing a seventh step of calling control means, and performing an eighth step of determining whether there is a compression code if there is no code compression or data conversion or data captured from the audio capture device, Performing a ninth step of writing a compressed code to a file when there is code compression or data conversion or data captured from the capture device, and performing code compression or data conversion or capture from the audio capture device; If there is no data first to determine whether compacted coded or data conversion or capture is completed
Step 11 is performed. If the code compression or data conversion or capture is not completed, the process returns to the fourth step. If the code compression or data conversion or capture is completed, a file is closed. And executing a twelfth step of issuing a termination request to the code compression control to release the acquired memory. 6. The code compression / data conversion apparatus according to claim 5, wherein The gist of the invention described in claim 7 of the present invention resides in that the original image transfer control means executes a twenty-first step of adding a capture frame number and executes a twenty-second step of determining whether or not to temporarily stop. If it is not the pause, the process proceeds to a step 26 for determining whether or not the video compression buffer can store the code-compressed or data-converted or captured original image. If it is the pause, it is determined whether or not to resume. Performing the twenty-third step, and if not restarting, proceed to a twenty-fifth step of returning a predetermined response to the application; otherwise, determine whether the capture frame number is greater than or equal to the restart frame number.
Performing the fourth step, and if the capture frame number is not equal to or greater than the restart frame number, executing a twenty-fifth step of returning a predetermined response to the application and terminating the processing; and if the capture frame number is equal to or greater than the restart frame number. Executes a 26th step of judging whether the compressed image or data conversion or the captured original image can be stored in the video capture buffer, and if the capture frame number is not equal to or larger than the restart frame number, the pause / restart Proceeding to a twenty-ninth step of activating the control means, if the compressed video or data converted or captured original image can be stored in the video capture buffer, the compressed video or data converted or captured original image is stored in the video capture buffer. Case Performing a twenty-seventh step, and performing a twenty-eighth step of notifying the video code conversion control means that there is code compression or data conversion or captured data. The code compression / data conversion apparatus according to claim 5, wherein the restart control means is activated. The gist of the invention described in claim 8 of the present invention is as follows.
The source audio transfer control means executes a thirty-first step of adding a capture sample number, and executes a thirty-second step of determining whether or not a pause has occurred. Alternatively, the process proceeds to a step 36 for judging whether or not the data conversion or the captured original image can be stored. In the case of the pause, a step 33 for judging whether or not the image is to be resumed is executed. To the 35th step of returning to the application, and in the case of the resumption, performing a 34th step of determining whether or not the capture sample number is equal to or greater than the resumption sample number, and the capture sample number is not greater than or equal to the resumption sample number. In this case, a predetermined response is returned to the application. Performing the process and terminating the process. If the captured sample number is equal to or larger than the resumed sample number, it is determined whether the audio capture buffer can store a code-compressed or data-converted or captured original image. And
Activating the pause / restart control means if the compressed image or data converted or the captured original image cannot be stored in the audio capture buffer.
Proceeding to a ninth step, if a code-compressed or data-converted or captured original image can be stored in the audio capture buffer, a code-compressed or data-converted or captured original image is stored in the audio capture buffer in a 37th step And performing a thirty-eighth step of notifying the video code conversion control means that there is code compression or data conversion or captured data. If the process proceeds to the thirty-ninth step, the pause / resume control means is executed. The code compression / data conversion device according to claim 5, wherein The gist of the invention described in claim 9 is that the video code conversion control means executes a forty-first step of determining whether or not there is data stored in the video capture buffer. Performing a forty-second step of extracting the original image from the buffer, performing a forty-third step of adding the number of compressed frames, performing a forty-fourth step of video-compressing the original image, and storing the generated video code in the video code buffer. Execute the forty-fifth step,
Thereafter, a 46th step of determining whether or not there is a request for suspension from the interface control means is executed,
If there is no request for suspension from the interface control means, the process returns to the forty-first step. If there is a request for suspension from the interface control means, it is determined whether there is a request for resumption from the interface control means. Performing the step and returning to the beginning of the forty-seventh step if there is no request for resumption from the interface control means, and judging whether or not to end compression if there is a request for resumption from the interface control means.
6. The code according to claim 5, wherein the process is performed in eight steps, and if there is no request for resumption, the process returns to step 41, and if there is a request for resumption from the interface control means, the process is terminated. The compression / data converter resides. The gist of the invention described in claim 10 is that the audio code conversion control means executes a fifty-first step of determining whether or not there is data stored in the audio capture buffer. Performing a 52nd step of extracting the original image from the buffer, performing a 53rd step of adding the number of compressed samples, and performing a 54th step of audio-compressing the original audio;
Performing a fifty-fifth step of storing the generated audio code in the audio code buffer, and subsequently performing a fifty-sixth step of determining whether there is a request for a pause from the interface control means; If there is no request for suspension, the process returns to the 51st step. If there is a request for suspension from the interface control means, it is determined whether there is a request for resumption from the interface control means.
Performing the step, returning to the beginning of the fifty-seventh step when there is no request for resumption from the interface control means, and performing a fifty-eighth step for determining whether or not to end compression when there is a request for resumption from the interface control means. The code compression / data conversion apparatus according to claim 5, wherein the code compression / data conversion apparatus performs the compression and returns to the fifty-first step if the compression is not completed, and ends the processing if the compression is completed. The gist of the invention described in claim 11 is that the pause / resume control means executes a 61st step of temporarily stopping video compression and executes a 62nd step of temporarily stopping audio compression. Performing a 63rd step of determining whether the load has decreased; if the load has not decreased, the process returns to the 63rd step; if the load has decreased, the time is calculated from the number of compressed frames up to the present. Performing a sixty-fourth step of calculating the amount, followed by a sixty-fifth step of calculating the amount of time from the number of compressed samples to date;
Thereafter, a 66th step of calculating a difference in the amount of time from the number of compressed frames and the number of compressed samples is performed, and thereafter, a restart frame of video compression (video code compression) is performed based on the difference in the amount of time and the current capture frame number. Performing a 67th step of calculating a number, followed by a 68th step of calculating a restart sample number of audio compression (audio code compression) from the difference in the amount of time and the current number of captured samples; A 69th step of discarding all the original images stored in the video capture buffer is performed, and a 70th step of restarting video compression (video code compression) is performed. Performing a 71st step of discarding all data and executing a 72nd step of restarting audio compression (audio code compression). To claim 5 which consists in compacted coded / data conversion apparatus according. Further, claim 1 of the present invention
The gist of the invention described in claim 2 is that, when the video compression or the audio compression cannot be continued, the interface control means suspends the video code conversion control means and the audio code conversion control means and transfers the video code conversion control means and the audio code conversion control means from the application. A predetermined response is returned to the application without compressing the video and audio capture data to be compressed, and video compression (video code compression) and audio compression (audio code compression) are performed so that the time difference between the video code and the audio code is reduced. The code compression / data conversion apparatus according to any one of claims 6 to 12, wherein the real time code compression is restarted by changing the restart time of (en). Further, claim 1 of the present invention
The gist of the invention described in claim 3 is that the interface control means temporarily stops the video code conversion control means and the audio code conversion control means when video compression or audio compression cannot be continued, and A predetermined response is returned without processing the passed original image and original sound, and thereafter, the difference in the amount of time between the generated video code and the audio code is checked, and if the video code is larger, the difference is reduced. Wherein the restart time of audio compression (audio code compression) is advanced, and the restart time of audio compression (audio code compression) is delayed so as to reduce the difference when the number of video codes is smaller. Item 12 is the code compression / data conversion device. The gist of the invention described in claim 14 of the present invention is a code compression / data conversion method for code-compressing an image and a sound on a system capable of operating a plurality of programs and recording the image and sound on a storage medium in real time. Returning a predetermined response to the application without compressing or converting data or compressing the captured image data while video compression cannot be continued; and performing code compression or data conversion or capturing while the audio compression cannot be continued. Returning a predetermined response to the application without compressing the audio data; restarting video compression (video code compression) at a time that reduces the time difference between the video code and the audio code; Restarting audio compression at a time that reduces the amount of time difference between Has an interface control step to collectively process image compression processing so that the application does not need to be directly involved in the compression processing. Execution of other programs increases the load on the system, making it impossible to continue real-time compression. When
Code compression / data, wherein the application automatically suspends real-time compression without any processing, and then automatically resumes real-time code compression with AV synchronization. It consists in a conversion method.

[0025]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention relates to a code compression / data conversion apparatus for code-compressing an image and a sound on a system capable of operating a plurality of programs and recording the resulting data in a storage medium in real time by executing another program. When the load on the system rises and real-time compression cannot be continued, the application side automatically suspends real-time compression without performing any processing, and then automatically synchronizes with the real-time real-time code. It is intended to provide a configuration capable of restarting compression.

(First Embodiment) Hereinafter, a first embodiment of the present invention will be described.
An embodiment will be described in detail with reference to the drawings. As described later with reference to FIG. 7, the interface control unit 42 collectively processes the image compression processing so that the application 41 does not need to directly participate in the compression processing.

When the video compression (video code compression) or the audio compression (audio code compression) cannot be continued, the interface control means 42 controls the video code conversion control means 10 and the audio code conversion control means 2.
0 is temporarily stopped, and a predetermined response is returned to the application 41 without compressing the video and audio capture data passed from the application 41. Next, real-time code compression is restarted by changing the restart time of video compression (video code compression) and audio compression (audio code compression) so that the difference in the amount of time between the video code and the audio code is reduced. Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, MPEG (Moving Picture E)
(xpartsGroup) compression method will be described with reference to the drawings.

FIG. 1 is a configuration diagram of a code format conforming to MPEG video. Referring to FIG. 1, in the present embodiment, video data is composed of one or more video sequences (Video Sequence), and ends with a video sequence end code.

The video sequence includes a video sequence header (Video Sequence Head).
r) and one or more GOPs (Group Of Pic)
cure). A GOP is composed of one or more pictures, and one picture indicates one image. At the head of the picture, a picture header including information such as a picture type is placed.

The pictures include an I picture composed of only intra-frame codes, a P picture composed only of forward inter-frame codes, and a B picture composed of preceding and succeeding bi-directional inter-frame codes. There are three types. Such a picture is composed of a plurality of slices divided into arbitrary regions.

The slice is composed of a plurality of macroblocks arranged in order from left to right or from top to bottom. The macroblocks are roughly classified into two types: intra blocks, which are intra-frame codes, and inter blocks, which are forward or bidirectional inter-frame codes.

The I picture is composed of only intra blocks, but P pictures and B pictures may include not only inter blocks but also intra blocks.

The macro block is 16 × 16
Six luminance components (Y1, Y2, Y3, Y4) obtained by further dividing the dot block into 8 × 8 dot blocks, and six color difference components (Cb, Cr) of the 8 × 8 dot block in the region matching the luminance component Block. The block of 8 × 8 dots is the minimum unit of encoding.

FIG. 2 is a configuration diagram of a code format conforming to MPEG audio. Referring to FIG. 2, in the present embodiment, audio data is composed of one audio sequence, and the audio sequence is one audio sequence.
AAUs (AudioAccess Units)
Audio decoding unit).

The one AAU includes a synchronization word,
A including information such as bit rate and sampling frequency
It is composed of an AU header and a compressed audio code. One AAU is the minimum unit of encoding.

FIG. 3 is a configuration diagram of a multiplex format of a video code and an audio code conforming to the MPEG system. Referring to FIG. 3, in the present embodiment, a video code / audio code is composed of one system stream, and the system stream is composed of a plurality of packs.

The pack includes a pack header, a system header, and one or more packets. Further, the system header stores information such as a bit rate.
It may or may not be attached to the subsequent packs. Packets include video packets and audio packets.

The video packet includes a packet header and a video code. When only the video code in the video packet is extracted and continued, a series of video sequences is formed.

The packet header of the video packet includes a packet start code, a code indicating a packet length (the total number of bytes of packet data immediately following the packet length), and a PTS (Presen) indicating a reproduction output time.
station time stamp) and a decoding time stamp (DTS) indicating a decoding time.
p) and the like.

In the above video code, since the I picture and the P picture are coded before the B picture, the order of decoding and the order of reproduction may be different. For this reason, two types of codes indicating the time are also provided for reproduction and decoding. If the decoding time and the reproduction time are different, both PTS and DTS are output. If the decoding time and the reproduction time are the same, only the PTS is output.

The audio packet includes a packet header and an audio code. A series of audio sequences is formed by extracting and continuing only the audio code in such an audio packet.

The packet header of the audio packet has a packet start code, a code indicating a packet length, and a PTS (Presentat) indicating a reproduction output time.
ion Time Stamp). Since the order of decoding and the order of reproduction are the same in the audio code, only the PTS is output.

If the PTS of the video packet and the PTS of the audio packet match, the codes are reproduced simultaneously.

In the following, images and sounds are reproduced in real time by MPE.
A description will be given of a compression operation when the compression is temporarily stopped during the G compression and resumed with AV synchronization. FIG. 7 shows a code compression / data conversion apparatus 1 according to the first embodiment of the present invention.
It is a functional block diagram for explaining 00.

Referring to FIG. 7, a code compression / data conversion apparatus 100 according to the present embodiment includes an application 41 operated by a user, an interface control means 42 for controlling a compression operation in accordance with a request from the application 41, 4, a video code conversion control means 10 for controlling video compression (video code compression) described later, an audio code conversion control means 20 for controlling audio compression (audio code compression) described later in FIG. 5, and FIG. Image / audio code conversion control means 30 for controlling system coding described later, video capture buffer 43 for temporarily storing code-compressed or data-converted or captured video data, and temporarily stored generated video codes Video code buffer 44 and code compression or data conversion or Audio capture buffer 46 for temporarily storing Yapucha audio data
, An audio code buffer 47 for temporarily storing the generated audio code, and a system buffer 45 for temporarily storing the system code.

The interface control means 42 includes an original image transfer control means 48 for transferring the code-compressed or data-converted or captured original image to the video code-conversion control means 10, and a code-compressed or data-converted or captured original sound. Audio code conversion control means 20
And a pause / resume control unit 50 for suspending and resuming code compression.

The application 41 receives the original image and the original image that have been code-compressed, converted, or captured, and passes them to the interface control means 42.
The system code passed to the interface control means 42 is written in a system code file.

The interface control means 42 stores the code-compressed or data-converted or captured original image in the video capture buffer 43, stores the code-compressed or data-converted or captured original audio in the audio capture buffer 46, The code conversion control means 10, the audio code conversion control means 20, and the image / audio code conversion control means 30 are controlled, and the generated codes are stored in the video code buffer 44 and the audio code buffer 4.
7 and the system buffer 45 and pass the code to the application 41.

When video compression (video code compression) or audio compression (audio code compression) cannot be continued, the interface control means 42 controls the video code conversion control means 10 and the audio code conversion control means 2.
0 is temporarily stopped, and a predetermined response is returned without processing the original image and original sound passed from the application 41. Thereafter, the difference between the amount of time between the generated video code and the audio code is checked. If the number of video codes is larger, the restart time of audio compression (audio code compression) is advanced so that the difference is reduced. When the number of video codes is smaller, the restart time of audio compression (audio code compression) is delayed so as to reduce the difference.

In this embodiment, since the interface control means 42 processes the suspension and resumption of the code compression in this manner, the application 41 does not need to perform any processing.

In the code compression / data converter 100 shown in FIG. 7, the application 41 receives the original image and the original sound and writes the generated code into a file. However, the application is not limited to this. The interface control means 42 may perform these two processes.

Next, the code compression / data conversion device 100
(Code compression / data conversion method) will be described.

FIG. 8 is a flow chart for explaining one process of the capture code compression processed by the application 41. Referring to FIG. 8, in the capture compression of the present embodiment, the video and audio capture devices are initialized (step S1), and the code compression control (video code conversion control means 10, audio code conversion control means 20) starts. A request is issued to acquire necessary memory (step S2), a file for recording the MPEG code is opened (step S3), and it is determined whether or not there is code compression or data conversion or data captured from the video capture device. (Step S
4) If not (N in step S4), the process proceeds to step S6. If so (Y in step S4), the original image transfer control unit 48 is called (step S4).
5).

Thereafter, it is determined whether or not there is code compression, data conversion, or captured data from the audio capture device (step S6).
Otherwise (N in step S6), the process proceeds to step S8. If so (Y in step S6), the original voice transfer control means 49 is called (step S7).

Thereafter, it is determined whether or not there is a compression code (step S8). If not (step S8)
N) proceeds to step S10. If so (Y in step S8), the compression code is written to the file (step S9), and it is determined whether or not the capture has been completed (step S10). If not (step S10)
N) returns to step S4. If so (Y in step S10), the file is closed (step S1).
1), code compression control (video code conversion control means 10,
It issues an end request to the audio code conversion control means 20) to release the acquired memory (step S12).

FIG. 9 is a flowchart for explaining one process of the original image transfer control means 48. Referring to FIG. 9, original image transfer control means 48 according to the present embodiment.
Adds the capture frame number (step S2
1) It is determined whether or not it is a temporary stop (step S22),
Otherwise (N in step S22), step S2
Proceed to 6.

If so (Y in step S22), it is determined whether or not to resume (step S23). If not (N in step S23), the process proceeds to step S25.

If so (Y in step S23), it is determined whether the capture frame number is equal to or greater than the restart frame number (step S24). If not (N in step S24), a predetermined response is sent to the application 41. Return (step S25) and end.

If so (Y in step S24), it is determined whether the video compression buffer 43 can store the code-compressed or data-converted or captured original image (step S26). If not (step S26) N) proceeds to step S29.

If so (Y in step S26), the compressed or data converted or captured original image is stored in the video capture buffer 43 (step S27), and the video code conversion control means 10 performs code compression or data conversion. It notifies that there is converted or captured data (step S28).

When the operation proceeds to step S29, the pause / restart control means 50 is started. Since this operates in a multitask, the operation is terminated without waiting for the suspension / resumption control unit 50 to terminate.

As described above, in the present embodiment, there is no need to stop the video capture operation even during the temporary stop, so that there is an effect that the video capture operation can be immediately resumed.

FIG. 10 is a flowchart for explaining one process of the original voice transfer control means 49. FIG.
, The original audio transfer control means 49 of the present embodiment adds the capture sample number (step S
31), and determine whether or not it is a temporary stop (step S3).
2) If not (N in step S32), the process proceeds to step S36.

If so (Y in step S32), it is determined whether or not to resume (step S33). If not (N in step S33), the process proceeds to step S35.

If so (Y in step S33), it is determined whether the capture sample number is greater than or equal to the restart sample number (step S34). If not (N in step S34), a predetermined response is sent to the application 41. Return (step S35) and end.

If so (Y in step S34), it is determined whether the audio compression buffer 46 can store the code-compressed or data-converted or captured original image (step S36). If not (step S36). N) proceeds to step S39.

If so (Y in step S36), the code compression or data conversion or the captured original image is stored in the audio capture buffer 46 (step S37), and the code compression or data conversion is performed by the audio code conversion control means 20. It notifies that there is converted or captured data (step S38).

When the operation proceeds to step S39, the pause / restart control means 50 is started. Since this operates in a multitask, the operation is terminated without waiting for the suspension / resumption control unit 50 to terminate.

As described above, in the present embodiment, there is no need to stop the audio capture operation even during the temporary stop, so that there is an effect that the operation can be immediately resumed.

In the processing shown in FIGS. 9 and 10, the pause is started when the data cannot be stored in the capture buffer. However, the present invention is not limited to this, and a code regarded as the same as the previous frame is inserted. Even when compression cannot be performed in real time, a pause may be started.

FIG. 11 is a flow chart for explaining one process of the video code conversion control means 10. FIG.
, The video code conversion control means 10 of the present embodiment determines whether or not there is data stored in the video capture buffer 43 (step S41). If there is no data (N in step S41), the process proceeds to step S41. Back to the top. If there is (Y in step S41), the original image is extracted from the video capture buffer 43 (step S4).
2) The number of compressed frames is added (step S43), and the original image is subjected to video compression (video code compression) (step S43).
44), the generated video code is stored in a video code buffer 44.
(Step S45).

Thereafter, the interface control means 42
It is determined whether or not there is a request for a temporary stop (step S46). If not (N in step S46), the process returns to step S41.

If so (Y in step S46), it is determined whether there is a request for resumption from the interface control means 42 (step S47). If not (N in step S47), the process returns to the beginning of step S47. .

If so (Y in step S47), it is determined whether or not the compression is completed (step S48). If not (N in step S48), the process returns to step S41. If so (Y in step S48), the process ends.

As described above, in the present embodiment, the pause / resume is performed in the video code conversion control means 10 in response to the request from the interface control means 42, so that the application 41 needs to perform the pause / resume processing. Absent. In addition, since the capture device is temporarily stopped while the initialization, the acquisition of the memory, and the opening of the file are performed, it is possible to immediately resume the operation.

FIG. 12 shows the audio code conversion control means 2
9 is a flowchart for explaining one process of the "0" process. Referring to FIG. 12, audio code conversion control means 20 of the present embodiment determines whether or not there is data stored in audio capture buffer 46 (Step S).
51), take out the original image from the audio capture buffer 46 (step S52), add the number of compressed samples (step S53), compress the original audio (audio code compression) (step S54), and generate the generated audio code. The data is stored in the audio code buffer 47 (step S55).

Thereafter, the interface control means 42
It is determined whether or not there is a request for suspension (step S56). If not (N in step S56), the process returns to step S51.

If so (Y in step S56), it is determined whether there is a request for resumption from the interface control means 42 (step S57). If not (N in step S57), the process returns to the beginning of step S57. .

If so (Y in step S57), it is determined whether or not compression has been completed (step S58). If not (N in step S58), the process returns to step S51. If so (Y in step S58), the process ends.

As described above, in the present embodiment, the pause / resume is performed in the audio code conversion control means 20 in response to the request from the interface control means 42, so that the application 41 needs to perform the pause / resume processing. Absent. In addition, since the capture device is temporarily stopped while the initialization, the acquisition of the memory, and the opening of the file are performed, it is possible to immediately resume the operation.

FIG. 13 is a flow chart for explaining one process of the suspension / resumption control means 50. FIG.
, The pause / resume control means 50 of the present embodiment temporarily suspends video compression (video code compression) (step S61) and pauses audio compression (audio code compression) (step S62). ), It is determined whether the load has decreased (step S63). If not (N in step S63), the process returns to step S63.
If so (Y in step S63), the amount of time is calculated from the number of compressed frames up to the present (step S6).
4).

For example, when the number of compressed frames is 1200 and the frame rate is 29.97 fps, the following calculation is performed. 1200 / 29.97 = 40.040 seconds

Thereafter, the amount of time is calculated from the number of compressed samples up to the present (step S65).

For example, if the number of compressed samples is 1786000
When the sampling frequency is 44.1 KHz, the following calculation is performed. 1786000/44100 = 40.498 seconds

After that, a difference in the amount of time is calculated from the number of compressed frames and the number of compressed samples (step S66).

In the above example, the following calculation is performed. 40.498-40.040 = 0.458 seconds

Thereafter, the restart frame number of the video compression (video code compression) is calculated from the difference in the amount of time and the current capture frame number (step S67).

For example, when the current capture frame number is 1229 frames, the following calculation is performed. 1229 + α × 29.97 = 1229 + 2.997 = 123.997 Rounded down to 1232 frames (α is an optimal value for each device, 0.1 in the example).

Thereafter, the restart sample number of audio compression (audio code compression) is calculated from the difference in the amount of time and the current number of captured samples (step S6).
8).

For example, if the current number of captured samples is 1
In the case of 808438, the following calculation is performed. 18084438 + 0.458 × 44100 + α × 44100 = 18084438 + 202197.8 + 4410 = 1833045.8 The result is rounded down to 183,046 samples (α is an optimum value for each apparatus, for example, 0.1).

Thereafter, the video capture buffer 43
Are discarded (step S69),
Resume video compression (video code compression) (step S
70), all original voices stored in the audio capture buffer 46 are discarded (step S71), and audio compression (audio code compression) is resumed (step S71).
72).

In the above calculation example, since the video code is smaller than the audio code, video compression (video code compression) is performed.
Compression of audio resume time (audio code compression)
However, when the video code is larger than the audio code, the restart time of the video compression (video code compression) is made later than the restart time of the audio compression (audio code compression).

As described above, in the present embodiment, since the difference in the amount of time between the video code and the audio code is reduced by shifting the restart time, there is an effect that the code compression can be restarted by synchronizing the video and the audio. .

In the processing shown in FIG. 13, the restart time is calculated after the load is reduced. However, the present invention is not limited to this. The restart time is calculated after waiting for a certain time. You may. Further, in the processing shown in FIG. 13, synchronization is achieved only by shifting the restart time.
Without being particularly limited to this, when the time difference between the original image and the original audio passed to the code compression control is large, a code assumed to be the same as the previous frame is inserted into the video code, and silence is added to the audio code. , Or the like, the difference may be reduced, and then the restart time may be shifted for synchronization. Alternatively, the video code may be framed in units of frames, and the audio code may be synchronized in units of AAUs by truncating the larger code in order to reduce the difference and then shift the restart time. Further, in the processing shown in FIG. 13, the difference in the amount of time is calculated from the number of compressed frames and the number of compressed samples. However, the present invention is not limited thereto, and the code amount and bit rate of the generated video code and audio code are calculated. , Or the time difference may be calculated from the time stamps of the generated video and audio packets of the system code.

FIG. 14 is a view for explaining one process of the video code and the audio code generated in FIG. FIG.
As described above, immediately after the pause, the video code has a frame count of 29.97 fps at 1200 frames, so the amount of time is 40.040 seconds, and the audio code has a sample count of 1786000 samples and a sampling frequency of 44.1 KHz. , The amount of time is 40.498 seconds, and the difference between the amount of time between the video code and the audio code is 0.458 seconds.

As described with reference to FIG. 13, since the video restart frame number is 1232 frames and the audio restart sample number is 183046 samples, the time amount of the preceding video code is calculated as follows. (18333046/44100)-(1232 / 29.97) = 41.565-41.107 = 0.458 sec.

As described above, since the time difference between the video code and the audio code immediately after the pause is 0.458 seconds, the difference is 0.458−0.458 = 0 seconds, and the audio compression (audio code compression) is performed. ) Resumes when the difference disappears, so that video and audio can be synchronized.

As described above, according to the present embodiment, the following effects can be obtained. First, the first effect is that even if the application 41 does not perform processing, the image / speech code compression is automatically and easily suspended when the system load increases, and the image / audio code compression is automatically stopped when the system load decreases. -It is possible to construct a system that restarts speech code compression. The reason is that the code compression control device (not shown)
This is because the control of the code compression of the image / sound is integrated so that the application 41 does not need to directly participate in the compression processing.

The second effect is that code compression can be resumed immediately after the load decreases. The reason is that compression is paused without stopping the capture operation,
This is because code compression can be restarted without performing initialization of a capture device that takes a long processing time, or acquiring memory and opening a file for code compression.

(Second Embodiment) Hereinafter, a second embodiment of the present invention will be described in detail with reference to the drawings. Note that the same parts as those already described in the above embodiment are denoted by the same reference numerals, and redundant description will be omitted.

FIG. 15 is a functional block diagram of a second embodiment for executing video code conversion (one form of image code conversion) for converting an image size. In the video code conversion according to the present embodiment, a configuration for converting a video code of only an I picture (one form of an image code) is shown for simplicity of description.

Referring to FIG. 15, a video code conversion control means 10 according to the present embodiment comprises a decoding means 151 for decoding a video code and an inverse quantization means 152 for performing an inverse quantization.
An inverse DCT unit 153 for converting a video code from a spatial frequency to an original (temporal space), an image size converting unit 154 for converting an image size, a DCT unit 155 for converting a video code from a temporal space to a spatial frequency, And a variable length coding unit 157 for performing variable length coding.

[0010] In the video code conversion executed by the video code conversion control means 10 of the present embodiment, the video code is converted into image data, and then the image size is changed and compressed to a new video code. In addition, the video code is decoded by the decoding means 151, inversely quantized by the inverse quantization means 152, and
The image data is returned to the T means 153. Next, the image size of the image data is converted by the image size
The data is converted into a spatial frequency by the T means 155, quantized by the quantization means 156, and subjected to variable length coding by the variable length coding means 157 to output a new video code.

(Third Embodiment) Hereinafter, a third embodiment of the present invention will be described in detail with reference to the drawings. Note that the same parts as those already described in the above embodiment are denoted by the same reference numerals, and redundant description will be omitted. FIG. 16 is a functional block diagram of the third embodiment for executing audio code conversion (one form of voice code conversion).

Referring to FIG. 16, the audio code conversion control means 20 of the present embodiment includes a frame decomposition means 161 for decomposing an audio code (one form of a voice code) for audio code conversion, and a decoding and inverse quantization. 162, an inverse mapping means 163 for performing an inverse mapping, an audio frequency converting means 164 for converting an audio frequency,
Frequency band mapping means 16 for generating the second frequency band mapping
5 and quantization encoding means 1 for performing linear quantization and encoding
66, a frame forming means 167 for creating compressed data by adding additional information to the encoded data, and a psychological hearing processing means 168 for performing psychological hearing processing.

[0010] In the audio code conversion executed by the audio code conversion control means 20 of the present embodiment, the audio code is converted into audio data, the frequency is converted, and the audio code is compressed into a new audio code. Further, the audio code is decomposed by the frame decomposing means 161 and
Perform decoding and inverse quantization, and perform inverse mapping by inverse mapping means 163 to return to audio data. Next, the audio frequency is converted by the audio frequency conversion means 164, and the audio is decomposed into 32 sub-band signals by the frequency band mapping means 165.
The scale factor is calculated for each subband signal, the dynamic range is adjusted, and the psychological auditory processing means 16 is calculated.
8, fast Fourier transform is performed on the speech, and the result is used to calculate psychological auditory masking, calculate bit allocation for each subband, and perform quantization and coding according to the bit allocation determined by the quantization coding means 166. Then, a header and auxiliary information are added to the sub-band signal quantized and encoded by the frame forming means 167, and the sub-band signal is shaped into a bit stream and output as a new audio code.

(Fourth Embodiment) Hereinafter, a fourth embodiment of the present invention will be described in detail with reference to the drawings. Note that the same parts as those already described in the above embodiment are denoted by the same reference numerals, and redundant description will be omitted. FIG. 17 is a functional block diagram of the fourth embodiment for executing conversion control without code compression.

Referring to FIG. 17, the image / audio code conversion control means 30 of the present embodiment is constituted mainly by a code analysis processing means 171 for analyzing a video code or an audio code and a thinning processing means 172. I have.

In the code conversion executed by the image / audio code conversion control means 30 of the present embodiment, the internal data of the data is analyzed by the code analysis processing means 171 without decoding the video code or the audio code, and (Decimation processing is performed by the decimation processing means 172 to convert to a desired low bit rate (color, frame, pixel, etc. for images, sampling frequency, number of bits per sample, number of channels, etc. for audio)) And output the new video code or audio code.

In the above-described embodiment, the description has been made with respect to only captured data. However, the present invention is not limited to this, and data used in digital broadcasting or the like (particularly, digitally compressed data) may be used. Data), and transcoding of compressed data. When the present invention is applied to data used in digital broadcasting or the like (particularly, data that has been digitally compressed), the capture processing may be replaced with code compression or data conversion processing. Further, it is apparent that the present invention is not limited to the above-described embodiment, and that the above-described embodiment can be appropriately modified within the scope of the technical idea of the present invention. Further, the number, position, shape, and the like of the constituent members are not limited to the above-described embodiment, but are suitable numbers for implementing the present invention,
Position, shape, etc. In each drawing, the same components are denoted by the same reference numerals.

[00111]

Since the present invention is configured as described above, the following effects can be obtained. The first effect is that even if the application does not process, the video / audio code compression is automatically and easily suspended when the system load increases, and the image / audio code compression is automatically stopped when the system load decreases.
That is, it is possible to construct a system for restarting speech code compression. The reason is that the control of image / sound code compression is centralized in a code compression control device (not shown) so that the application does not need to directly participate in the compression processing.

The second effect is that code compression can be restarted immediately after the load decreases. The reason is,
This is because the compression is temporarily stopped without stopping the capture operation, so that the code compression can be resumed without performing the initialization of the capture device, which takes a long time, or acquiring the memory and opening the file for the code compression.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a code format conforming to MPEG video.

FIG. 2 is a configuration diagram of a code format conforming to MPEG audio.

FIG. 3 is a configuration diagram of a multiplex format of a video code and an audio code conforming to the MPEG system.

FIG. 4 is a configuration diagram of a video code conversion control unit that performs image compression.

FIG. 5 is a configuration diagram of audio code conversion control means for performing audio compression.

FIG. 6 is a diagram showing a structure of an image / audio code conversion control means for multiplexing a video code and an audio code.

FIG. 7 is a diagram illustrating code compression / coding according to the first embodiment of the present invention.
It is a functional block diagram for explaining a data conversion device.

FIG. 8 is a flowchart for explaining one process of capture code compression processed by an application.

FIG. 9 is a flowchart illustrating one process of an original image transfer control unit.

FIG. 10 is a flowchart for explaining one process of an original voice transfer control means.

FIG. 11 is a flowchart illustrating one process of video code conversion control means.

FIG. 12 is a flowchart illustrating one process of audio code conversion control means.

FIG. 13 is a flowchart illustrating one process of a suspension / resumption control unit.

14 is a diagram illustrating one process of the video code and the audio code generated in FIG.

FIG. 15 is a functional block diagram of a second embodiment for executing video code conversion for converting an image size.

FIG. 16 is a functional block diagram of a third embodiment for executing audio code conversion.

FIG. 17 shows a fourth example of performing conversion control without code compression.
It is a functional block diagram of an embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Video code conversion control means 20 ... Audio code conversion control means 30 ... Image / audio code conversion control means 41 ... Application 42 ... Interface control means 43 ... Video capture buffer 44 ... Video code buffer 45 ... System buffer 46 ... Audio capture buffer 47 audio code buffer 48 original image transfer control means 49 original sound transfer control means 50 pause / restart control means 100 code compression / data conversion device 151 decoding means 152 inverse quantization means 153 inverse DCT means 154 image size converting means 155 DCT means 156 quantizing means 157 variable length coding means 161 frame decomposing means 162 reconstructing means 163 inverse mapping means 164 voice frequency converting means 165 frequency band mapping Image means 166 ... Quantization coding means 167 ... Frame forming means 168 ... Psychological hearing processing means 171 ... Code analysis processing means 172 ... Thinning processing means

Continued on front page F-term (reference) 5C053 GB11 GB37 JA07 JA22 KA03 KA24 5C059 KK35 MA00 MA05 MA14 MA23 MC11 ME01 NN01 NN21 PP05 PP06 PP07 PP16 RB10 RC04 RC07 RC32 RC37 SS06 SS12 TA00 TB04 TC15 TC18 TD11 UA02 UA0 UA32 A39 5 BA16 BB09 BC16 BC26 BD03

Claims (14)

[Claims] 1. A code compression / data conversion apparatus for code-compressing or data-converting an image and / or sound on a system capable of operating a plurality of programs and recording or distributing the data in real time. A code compression / data conversion device having an automatic pause or automatic restart function of compression or data conversion. 2. A code compression apparatus according to claim 1, further comprising means for returning a response to the application without code compression or conversion of the image or audio data while code compression or data conversion cannot be continued. /
Data converter. 3. The code compression / data conversion according to claim 1, further comprising means for restarting code compression or data conversion at such a time as to reduce a difference in time amount between the image code and the audio code. apparatus. 4. An interface control unit that collectively processes code compression or data conversion so that the application does not need to directly participate in code compression or data conversion processing. The system according to any one of claims 1 to 3, wherein when the load on the system increases and real-time processing cannot be continued, control from the application side or the operating system is not required. A code compression / data conversion apparatus as described in the above. 5. The application operated by a user, the interface control means for controlling a compression operation according to a request from the application, a video code conversion control means for controlling video compression, and an audio code conversion for controlling audio compression. Control means, image / audio code conversion control means for controlling system coding, video capture buffer for temporarily storing code compression or data conversion or captured video data, and temporarily storing generated video codes A video code buffer, an audio capture buffer for temporarily storing code compression or data conversion or captured audio data, an audio code buffer for temporarily storing generated audio codes, and a system code for temporarily storing system codes. An original image transfer control unit that passes a code compressed or data converted or captured original image to the video code conversion control unit; and a code compression or data converted or captured image buffer. An original audio transfer control unit for passing an original audio to the audio code conversion control unit; a pause / resume control unit for temporarily stopping and restarting code compression; the application includes code compression or data conversion or capture; Received the original image and audio,
The interface control means writes the system code passed to the interface control means to a system code file, and the interface control means stores the original image obtained by code compression or data conversion or capture in the video capture buffer. The code compression or data conversion or the captured original audio is stored in the audio capture buffer, and the video code conversion control means, the audio code conversion control means, and the image / audio code conversion control means are controlled, and the generated The code compression / code compression method according to claim 4, wherein a code is stored in the video code buffer, the audio code buffer, or the system buffer, and the code is passed to the application.
Data converter. 6. The application executes a first step of initializing a video and audio capture device, executes a second step of issuing a start request to a code compression control to acquire necessary memory, and executes an MPEG process.
Performing a third step of opening a file for recording the code, performing a fourth step of determining whether there is code compression or data conversion or captured data from the video capture device, and performing a code compression from the video capture device. Performing a fifth step of calling the original image transfer control means if there is data converted or converted or captured, and an audio capture device if there is no code compression or data converted or captured data from the video capture device Performing a sixth step of determining whether there is data that has been code-compressed or converted or captured from, and if there is code-compressed or converted or captured data from the audio capture device, Performing a seventh step of calling the original voice transfer control means, and performing an eighth step of determining whether there is a compression code if there is no code compression or data conversion or captured data from the audio capture device. Performing a ninth step of writing a compressed code to a file when there is code compression or data conversion or captured data from the audio capture device, and performing code compression or data conversion or capture from the audio capture device. If there is no data, execute a tenth step of determining whether code compression or data conversion or capture is completed, and return to the fourth step if code compression or data conversion or capture is not completed. The code compression When the encoding, data conversion, or capture is completed, the eleventh step of closing the file is executed, and the twelfth step of issuing an end request to the code compression control to release the acquired memory is executed. The code compression / data conversion device according to claim 5. 7. The original image transfer control means executes a twenty-first step of adding a capture frame number, and executes a twenty-second step of determining whether or not a pause has occurred. It is determined whether the buffer can store the code-compressed or data-converted or captured original image.
Proceed to the step, and if it is the temporary stop, determine whether to resume the second
While executing the three steps, proceeding to a twenty-fifth step of returning a predetermined response to the application if the resumption is not the resumption, and determining the twenty-fourth step of determining whether the capture frame number is equal to or greater than the resumption frame number if the resumption is the resumption. Executing, and if the capture frame number is not equal to or greater than the restart frame number, executing a twenty-fifth step of returning a predetermined response to the application, and terminating the process. Performing a twenty-sixth step of judging whether the compressed buffer or the data conversion or the captured original image can be stored in the capture buffer, and if the capture frame number is not equal to or greater than the restart frame number, the pause / restart control means Proceed to the 29th step to start, If the original image captured by code compression or data conversion or captured can be stored in the video capture buffer, a 27th step of storing the compressed image or data converted or captured original image in the video capture buffer is performed, A step 28 of informing the video code conversion control means that there is code compression or data conversion or captured data, and activating the pause / resume control means when the process proceeds to the 29th step. The code compression / data conversion device according to claim 5, wherein 8. The original voice transfer control means executes a thirty-first step of adding a capture sample number, and executes a thirty-second step of determining whether or not the audio capture is to be paused. The process proceeds to a step 36 in which it is determined whether or not the code compression or data conversion or the captured original image can be stored in the buffer.
In addition to executing the three steps, proceeding to the 35th step of returning a predetermined response to the application if the restart is not the above, and the 34th step of determining whether or not the capture sample number is equal to or larger than the restart sample number in the case of the restart Executing the 35th step of returning a predetermined response to the application if the capture sample number is not equal to or greater than the restart sample number, and terminating the process. If the capture sample number is equal to or greater than the restart sample number, Performing a 36th step of judging whether the code compression or data conversion or the captured original image can be stored in the capture buffer and not storing the code compression or data conversion or the captured original image in the audio capture buffer; If Proceeding to a thirty-ninth step of activating the pause / restart control means, the code compression or data conversion or the captured original image if the compressed or data converted or captured original image can be stored in the audio capture buffer. In the audio capture buffer, and a thirty-eighth step of notifying the video code conversion control means that there is code compression or data conversion or captured data. 6. The code compression / data conversion device according to claim 5, wherein the pause / resume control unit is activated when the process proceeds. 9. The video code conversion control means executes a 41st step of determining whether there is data stored in the video capture buffer, and executes a 42nd step of extracting an original image from the video capture buffer. And performing a forty-third step of adding the number of compressed frames, performing a forty-fourth step of video-compressing the original image, and performing a forty-fifth step of storing the generated video code in the video code buffer. Performing a 46th step of determining whether there is a request for a pause from the interface control means;
If there is no request for suspension from the interface control means, the process returns to the forty-first step. If there is a request for suspension from the interface control means, it is determined whether there is a request for resumption from the interface control means. Performing the step, returning to the beginning of the step 47 when there is no request for resumption from the interface control means, and performing a step 48 for determining whether or not to end compression when there is a request for resumption from the interface control means. 6. The method according to claim 5, wherein the step is executed, and if there is no request for resumption, the process returns to the forty-first step, and if there is a request for resumption from the interface control means, the process is terminated.
2. A code compression / data conversion device according to claim 1. 10. The audio code conversion control means executes a fifty-first step of determining whether there is data stored in the audio capture buffer, and retrieves an original image from the audio capture buffer.
Performing the two steps, performing a fifty-third step of adding the number of compressed samples, performing a fifty-fourth step of audio-compressing the original voice, and performing a fifty-fifth step of storing the generated audio code in the audio code buffer. Thereafter, a 56th step of determining whether or not there is a request for suspension from the interface control means is executed, and if there is no request for suspension from the interface control means, the process returns to the 51st step, When there is a request for suspension from the control means, a 57th step of judging whether or not there is a request for resumption from the interface control means is executed. To the beginning of the above, when there is a restart request from the interface control means Run the 58th step of determining whether the compression end,
The code compression / data conversion apparatus according to claim 5, wherein if the compression is not completed, the process returns to the fifty-first step, and if the compression is completed, the process is terminated. 11. The pause / resume control means executes a 61st step of temporarily stopping video compression, executes a 62nd step of temporarily stopping audio compression, and determines whether the load has decreased. Performing step 63, if the load is not reduced, return to the 63rd step, and if the load is reduced, perform a 64th step of calculating the amount of time from the number of compressed frames up to the present, Thereafter, a 65th step of calculating the amount of time from the number of compressed samples up to the present is performed, and then a 66th step of calculating the difference of the amount of time from the number of compressed frames and the number of compressed samples is performed. Performing a 67th step of calculating a restart frame number of video compression (video code compression) from the difference in the amount of time and the current capture frame number; Performing a 68th step of calculating a restart sample number of audio compression (audio code compression) from the number of pulls, and then performing a 69th step of discarding all original images stored in the video capture buffer, Performing a 70th step of restarting compression (video code compression);
The code according to claim 5, wherein a 71st step of discarding all original voices stored in the audio capture buffer is executed, and a 72nd step of restarting audio compression (audio code compression) is executed. Compression / data converter. 12. When the video compression or the audio compression cannot be continued, the interface control means suspends the video code conversion control means and the audio code conversion control means, and controls the video and audio signals passed from the application. Without compressing the captured data, return a predetermined response to the application,
Real-time code compression is restarted by changing the restart time of video compression (video code compression) and audio compression (audio code compression) so that the time difference between the video code and the audio code is reduced. The code compression / data conversion device according to any one of claims 6 to 12. 13. The interface control means, when video compression or audio compression cannot be continued, suspends the video code conversion control means and the audio code conversion control means, and the original image transferred from the application. And a predetermined response is returned without processing the original audio. Then, the difference in the amount of time between the generated video code and the audio code is checked, and if the video code is larger, the audio compression ( 13. The resumption time of audio compression (audio code compression) is advanced, and when the number of video codes is smaller, the resumption time of audio compression (audio code compression) is delayed so as to reduce the difference. Code compression / data conversion device. 14. A code compression / data conversion method for code-compressing images and sounds on a system capable of operating a plurality of programs and recording the data in a storage medium in real time, wherein code compression is performed while video compression cannot be continued. Returning a predetermined response to the application without encoding or data conversion or compressing the captured image data, and transmitting the predetermined response to the application without code compression or data conversion or compression of the captured audio data while audio compression cannot be continued. A step of returning, a step of restarting video compression (video code compression) at a time that reduces the time difference between the video code and the audio code, and a time of reducing the time amount of the video code and the audio code. Resuming audio compression; and An interface control step for consolidating and processing image compression processing so as not to be involved in the application, and when the load on the system increases due to the execution of another program and real-time compression cannot be continued, the application A code compression / data conversion method characterized by automatically suspending real-time compression without any processing, and then automatically resuming real-time code compression with AV synchronization thereafter. .