JPH10304360A - Method for encoding video/voice, device therefor and medium for recording coded program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly set an encoding parameter corresponding to the basic capability of a computer by determining the encoding parameter based on at least one of resolution, frame rate, and encoding parameter influencing processing performance. SOLUTION: This device is provided with an encoding means 101 and an encoding parameter decision means 102 including a resolution reference table 110. The encoding parameter decision means 102 determines resolution by using the included resolution reference table 110 from a designated frame rate and an encoding pattern, and outputs the encoding parameter including a parameter indicating the determined resolution to the encoding means 101. The encoding means 101 operates an encoding processing according to the encoding parameter so that encoding with higher resolution can be attained while satisfying a requested condition. The encoding pattern can be determined corresponding to the designated resolution by using the similar reference table.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video / audio coding method, a coding apparatus, and a coded program recording medium, and more particularly to a video / audio or video / audio control by software control using general-purpose computer resources. The present invention relates to an encoding method, an encoding device, and an encoded program recording medium for encoding in accordance with capture.

[0002]

2. Description of the Related Art Techniques for digitizing video and audio data, which are originally analog data, to obtain digital video data and audio data, are based on the ease of handling digital data recording, transmission, editing, duplication and transmission. It has become a remarkable area of diffusion and development. One of the advantages of digitization is that data can be easily compressed, and compression encoding is an important technique especially for recording and transmission. International standards have also been established for such compression encoding technology, and among them, the MPEG standard has become widespread as a general digital standard that can handle video and audio.

In recent years, computers and VLS
With the speeding up and cost reduction of semiconductor devices such as I, PCs called multimedia-capable personal computers have come on the market at low prices, and video that is compression-encoded digital data, which was conventionally performed by adding decoding hardware. Software can be easily reproduced on home and personal computers by software. Along with this, distribution of video and audio is also performed via the Internet and the like, and the use range of video and audio coded data conforming to standards such as MPEG is expanding.

[0004] On the other hand, generally, it is difficult to perform software processing on a home / personal personal computer for encoding (encoding) processing for generating coded data of video and audio, and dedicated hardware is added. Will be done. It is also possible to record once as a file and then perform encoding by software processing, but since conversion takes many times the processing time of video and audio input, conversion takes place. It's hard to say it's attractive software.

[0005] In order for ordinary PC users to be able to easily capture video and audio including video and create encoded data, video and audio must be captured using a capture board or sound board. Accordingly, it is desirable that real-time encoding processing by software be possible. This is a field in which development is desired with the development and spread of hardware. The current status in the field of video, audio, or video / audio coding is described below as “A video coding” and “B audio coding”.
A device according to the related art for performing “C video / audio coding” will be described.

A. 2. Description of the Related Art A video encoding device based on a conventional technique is to digitize a video including a still image or a moving image in real time and capture it in a computer, and to perform an encoding process along with the capture, which is an international standard for video compression. This is realized by using a personal computer expansion card or the like that encodes a video in real time using the system.

FIG. 58 is a block diagram showing a configuration of a video encoding device realized in a computer having such dedicated hardware. As shown in the figure, a video encoding device according to the related art includes an encoding unit 5001.
And encoding parameter determination means 5002, and inputs a video as input image data and outputs video encoded data. The encoding means 5001 includes the DCT processing means 50
03, quantization means 5004, variable length coding means 500
5, a bit stream generation unit 5006, an inverse quantization unit 5007, an inverse DCT processing unit 5008, and a predicted image generation unit 5009.

In the figure, an encoding means 5001 inputs video data consisting of a series of still images in which video has been digitized as input image data, performs an encoding process in accordance with set encoding parameters, and encodes the encoded data. Is output. Each still image data constituting the input image data is called a frame image. The coding parameter is provided from a coding parameter determination unit 5002, which will be described later, as an instruction of a coding type and a resolution. The encoding parameter determining unit 5002 determines an encoding type indicating an intra-frame encoding process or an inter-frame encoding process and a resolution, and outputs these to the encoding unit 5001 as encoding parameters.

[0009] Inside the encoding means 5001, first, the DCT means 5003 performs the DCT on the input image data.
(Discrete Cosine Transform) processing to output DCT transformed data, and then the quantization means 5004 performs quantization processing on the DCT transformed data to output quantized data;
Next, the variable-length encoding unit 5005 performs variable-length encoding on the quantized data, thereby creating compression-coded variable-length encoded data. The variable-length coded data is input to a bit stream generation unit 5006, and the bit stream generation unit 5006 outputs coded data as a coding result of the video coding device as a bit stream that can be transmitted or recorded. Is done.

The inverse quantization means 5007 includes a quantization means 50
The inverse DCT unit 5008 performs inverse quantization on the quantized data output from the inverse quantizing data and outputs the inversely quantized data. The inverse DCT process, which is the inverse process of the DCT process, is performed to output the inverse DCT transform data. The inverse DCT transform data is input to the predicted image generation unit 5003 and output as predicted image data. When the encoding process using the predicted image is performed according to the encoding parameter, the difference data between the predicted image data and the input image data is input to the DCT unit 5003, so that the encoding unit 5001 generates the frame. Inter-coding is performed.

The operation of the video coding apparatus according to the prior art configured as described above in the video coding process will be described below. First, prior to the encoding process,
The encoding parameter determination unit 5002 determines an encoding parameter for the encoding type and the resolution, and outputs this to the encoding unit 5001.

Generally, in compression coding, an intra-frame code for performing compression without redundancy on a still image of one frame (corresponding to one screen) based on its spatial correlation (correlation within a frame). And inter-frame coding for compressing a still image of a temporally close still image of, for example, a continuous frame without redundancy based on the temporal correlation (correlation between frames).

The video encoding apparatus according to the prior art basically performs intra-frame encoding. However, by also performing inter-frame encoding, encoded data with a high compression rate can be obtained. However, in order to perform inter-frame encoding, a predicted image is generated by a decoding process, which is a reverse process of the encoding, or a motion detection / motion compensation process, and a difference between the predicted image and the encoding target image is calculated. Since these processes are obtained, the processing load on the apparatus is increased by the amount of the need for these processes. For the generation of a predicted image when performing inter-frame encoding, forward prediction that performs prediction based on data processed immediately before, backward prediction that performs prediction based on data processed immediately after, and forward direction, Alternatively, one of bidirectional prediction in which backward prediction is performed is performed. In addition,
Intra-frame coding is denoted by “I”, forward prediction coding is denoted by “P”, and bidirectional prediction coding (including backward direction) is denoted by “B”.

[0014] Regarding the image resolution, "320
It is generally represented by the number of vertical and horizontal pixels (pixels) per screen such as “× 240” or “160 × 120”, and processing is performed at a high resolution, that is, as having many pixels in one screen. Although data with good reproduction image quality can be obtained, the processing load increases because the number of processing targets increases accordingly.

Further, in order to comply with the MPEG standard, it is necessary to perform input / output and transfer of data at a fixed transfer rate. In encoding processing, processing is performed so as to satisfy the transfer rate. Therefore, encoded data needs to be output. When a video is to be processed, the transfer rate is generally displayed at a frame rate represented by the number of frames per second.

Therefore, in consideration of the processing capability of the video encoding device, real-time processing accompanying video capturing is performed while satisfying this frame rate, and the compression quality is as high as possible and the reproduction quality is high (high-resolution It is desirable that the encoding parameters are set so that encoded data can be obtained.

In the video encoding apparatus according to the prior art, the encoding parameters are set in advance in consideration of these factors, and the encoding parameter means 5002 holds the set parameters and performs encoding. In this case, it can be output to the encoding means 5001.
As for the encoding type among the encoding parameters, a method of determining the encoding type based on information of an input video such as a scene change is described in "Image Encoding Apparatus (Japanese Patent Laid-Open No. 8-98185)". , Are disclosed.

Among the encoding parameters input to the encoding means 5001, a parameter indicating the resolution is input to the DCT processing means 5003 and used for processing. Also, the parameter indicating the encoding type is the DCT processing means 500
3 is the input image data itself,
This is used for controlling whether to make a difference from the predicted image.

The DCT processing means 5003, based on the resolution input from the coding parameter determining means 5002,
For the input frame image or difference data,
It performs DCT processing and outputs DCT conversion data. DC
In the T processing, generally, target data is divided into blocks of 8 pixels × 8 pixels, and two-dimensional discrete cosine transform is performed for each of the divided blocks. Next, the quantization unit 104 performs a quantization process on the DCT transformed data using a predetermined value, and outputs the quantized data. The quantization process is generally performed by a division process using the value of the quantization step (the above-defined value). Then, the variable length coding unit 105 performs variable length coding on the quantized data and outputs variable length coded data. The variable length coding is to reduce the entire data amount by allocating a code having a small number of bits to a high frequency code in a bit allocation in the coding process. The bit stream generating means 106 generates and outputs a bit stream as a coding result from the variable length coded data output from the variable length coding means 105 as a device output of the video coding device.

When inter-frame encoding is performed,
The following operation is performed. The inverse quantization means 5007 inversely quantizes the quantized data output from the quantization means 5004, and outputs inversely quantized data. Next, the inverse DCT processing means 5
008: DCT processing means 5
For each 8 × 8 pixel block divided by 003, 2
A two-dimensional inverse discrete cosine transform, which is an inverse process of the dimensional discrete cosine transform, is executed, and inverse DCT transform data is output. The predicted image generation means 5009 generates a predicted image based on the inverse DCT transform data, and outputs the generated predicted image. The DCT processing means 5003 receives the difference data between the input image data and the predicted image.

B. 2. Related Art Speech coding apparatus according to the prior art Regarding a speech coding method, a coding method based on a band division coding method conforming to the MPEG Audio method is applied to a wide band such as a human voice, music, a sound in a natural environment, and various sound effects. Used to encode general speech. In a high-performance multimedia personal computer or the like, it is also possible to perform real-time encoding processing on audio captured using a sound board that is generally equipped as standard. First of conventional speech coding devices
As an example, a description will be given of an example in which input speech is encoded by the above-described band division encoding scheme.

On the other hand, there is a method of applying psychoacoustic analysis as a speech coding method based on MPEG1 Audio. Originally, in an encoder conforming to MPEG1 Audio, the priority of allocating bits to each band is determined by using the psychoacoustic model in consideration of the limit of human hearing ability and the masking effect. This is for performing high-efficiency encoding according to the static and dynamic auditory characteristics of humans, but does not affect the data format of the MPEG1 Audio standard.
PEG1 Audio encoded data can be created. Further, as will be described later, the processing of the psychological hearing analysis has a large processing load, and as shown in the first example of the related art, it is possible to largely reduce the processing load on the CPU by omitting this processing. Becomes However, the reproduction sound quality is degraded because the psychoacoustic analysis is not applied. As a second example of speech encoding according to the related art, speech encoding that applies such psychological auditory analysis will be described.

B-1. First Example of Conventional Speech Coding Apparatus FIG. 59 is a block diagram illustrating a configuration of a first example of a conventional speech coding apparatus. As shown in the figure, the audio encoding device according to the first example includes an audio input unit 2551, an input audio sampling unit 2553, a band division unit 2555, an encoded bit allocation unit 2556, a quantization unit 2557, an encoding unit 2558, and It comprises an encoded data recording section 2559.

In the figure, a voice input unit 2551 inputs voice to be coded. Generally, sound is input from a microphone or as a line input. The input voice sampling unit 2553 is realized by an input function and a control program of the sound board, and performs a sampling process on the voice input by the voice input unit 2551. Band dividing section 2555 divides the band of the sampled data. Encoded bit allocation section 2556
Assigns coded bits to each of the bands divided by the band division unit 2555. Quantizer 2557
Performs a quantization process according to the number of coded bits allocated by the coded bit allocation unit 2556. Encoding unit 25
58 outputs the quantization value output from the quantization unit 2557 as encoded audio data. Each of 2555 to 2558 is realized by a CPU, a main memory, and a program of a computer. Encoded data recording section 2559
Is realized by a storage device such as a magnetic storage device, and a control program for the storage device, and records the output encoded data.

FIG. 60 is a flowchart of a conventional speech encoding method, FIG. 61 is a diagram for explaining sampling processing, and FIGS. 62 to 63 are diagrams for explaining band division. The operation of the speech encoding device according to the first example of the related art will be described below with reference to FIGS. 59 to 63 and the flow of FIG.

In step 1 of FIG. 60, the input audio sampling section 2553 samples the input audio signal at a preset sampling frequency fs to obtain sampling data. As shown in FIG. 61, the input voice is expressed as a graph showing the relationship between time and sound pressure. Sampling uses this input voice as time ts called the sampling period.
The sampling frequency ts and the sampling frequency fs have a reciprocal relationship as shown in the figure.

Step 2 and subsequent steps in FIG. 60 are processing centered on arithmetic processing performed by software under the control of the CPU. In step 2, the sampling data is divided into M frequency bands by the band dividing unit 2555. FIG. 62 is a conceptual diagram showing a case where audio data is used as a full-band input signal and divided into 12 bands, and it is assumed that 12 band signals from band 0 signal BPF0 to band 11 signal BPF11 are created. Is shown.
FIG. 63 is a diagram showing band signals divided into 12 bands as described above. In this figure, the band-divided signal is
Unlike FIG. 61, the sound pressure is represented not by time but by frequency.

In the case of MPEG audio, layers 1 to 3
Is defined, and the reproduction sound quality becomes good in the order of 1 → 2 → 3, but the required hardware performance increases and the hardware scale increases. Here, when performing audio coding adapted to Layer 1, the number p of input audio samples to be processed in one band division process is p = 32. The input audio sample is divided into 32 bands using 512 samples before and after the target 32 samples, and audio data for each band is output.

The M obtained by the band division in step 2
The band signal data are passed from band division section 2555 to quantization section 2557. On the other hand, in step 3, the coded bit allocation unit 2556 allocates coded bits to all of the M band signals. In step 4, the quantization unit 2557 performs quantization on the band signal data passed from the band division unit 2555 for each band according to the number of coded bits allocated by the coded bit allocation unit 2556. Go to the quantized value. Next, in step 5, the encoding unit 2558 encodes and outputs the quantized value, and the encoded data output unit 2559 records the output encoded data.

While the voice input continues, steps 1 to 5 are repeatedly performed, whereby the voice is continuously input, the real-time coding process is performed, and the coded data is output.
Keeps being recorded. When the voice input ends, the encoding process ends immediately. The encoded data stored in the storage device is stored as MPEG reproducible data. Alternatively, instead of recording and storing, encoded data can be transmitted and used via a network or the like. The above is the first example of the prior art, which is the audio encoding apparatus that obtains encoded data in real time with audio capture.

B-2. Second example of conventional speech coding apparatus FIG. 64 is a block diagram showing a configuration of a second example of the speech coding apparatus of the prior art. As shown in the figure, the audio encoding device according to the second example includes an audio input unit 2651, an input audio sampling unit 2653, a band division unit 2655, a quantization unit 2657, an encoding unit 2658, an encoded data recording unit 2
659, FFT unit 2660, psychological hearing analysis unit 2661,
And a coded bit allocation unit 2662. This device has a configuration in which an FFT unit 2660 and a psychological auditory analysis unit 2661 are added to the device of the first example.

In the figure, FFT (Fast Fourier Transform)
The unit 2660 performs Fourier transform processing on the signal so that psychological auditory analysis can be performed. The psychological hearing analysis unit 2661 compares the signal processed by the FFT unit 2660 with the minimum audible limit and analyzes the masking effect. The coded bit allocation unit 2662 allocates the coded bits based on the analysis result of the psychological auditory analysis unit 2661 so that the allocation of the coded bits to the signal audible to the human ear is relatively increased. Audio input unit 2651, input audio sampling unit 2653, band division unit 2655, quantization unit 2657, encoding unit 265
8, and the coded data recording unit 2659 are the same as those of the first example 2551 to 2555 and 2557 to 2559, and thus description thereof is omitted.

FIG. 65 is a flowchart of MPEG1 Audio encoding, and FIG. 66 is a diagram showing a minimum audible limit. Hereinafter, the operation of the speech encoding device of the second example will be described.
This will be described with reference to FIGS. Steps 1 and 2 in the flow of FIG. 65 are performed in the same manner as in the first example, and a signal divided into M bands is obtained. As a general example, it is assumed that M = 32 band signals are obtained.
As in the first example, the band signal is passed from the band division unit 55 to the quantization unit 57.

On the other hand, in step 3, the FFT unit 2660
After the sampled input voice data is divided into L bands by fast Fourier transform (FFT) processing, it is passed to a psycho-auditory analysis unit 2661, and the psycho-auditory analysis unit 2661 analyzes the L signals. For example, in the case of following the layer 1 of the MPEG audio, 512 sampling data are used, but the FFT unit 2660 divides the data into L = 256 bands by the fast Fourier transform process. In the case of Layer 2, since 512 bands are output using 1024 samples, the processing load increases accordingly.

The psycho-aural analysis unit 2661 compares each band signal with a minimum audible limit, which is a limit level that cannot be heard by human ears, as shown in FIG. FIG.
6 is shown as a division into 32 bands, even when the number of divisions is increased to 256 or the like as in this example, the graph of the minimum audible limit is the same, and the same range as shown in FIG. Is subdivided on the horizontal axis (band).

According to the analysis by the psychological auditory analysis unit 2661, no bit is allocated in the subsequent processing to the band determined to be less than the minimum audible limit, and accordingly, more bits are allocated to other bands. Become.

In human hearing, a relatively small sound, that is, a signal with a small sound pressure, cannot be heard when there is a large sound that is close in frequency or time, that is, a signal with a large sound pressure. It is recognized that there is a masking phenomenon. Therefore, the psychological hearing analysis unit 266
1 examines the relationship between signals in each band and adjacent signals, and detects signals that are masked (inaudible) by the masking phenomenon. Regarding the signal detected here, no bits are allocated in the subsequent processing, and accordingly, more bits are allocated to other bands.

In step 5 of the flow in FIG. 65, the coded bit allocating unit 2657
Is performed according to the result of the analysis. Here, the analysis result of the L band is assigned to the M band. Therefore, bits are not assigned to a signal that cannot be heard or hardly heard by human ears, and more bits are assigned to a signal that is well heard. Step 6 and subsequent steps are the same as in the first example, and by repeating steps 1 to 7,
Speech coding accompanying speech input is performed.

In this manner, more coded bits are allocated to speech that is more audible to human hearing, and thus, in MPEG audio speech encoding incorporating psychological auditory analysis, encoding with good reproduction sound quality is achieved. Voice data can be obtained.

C. Video / Audio Coding Apparatus According to Conventional Technique FIG. 67 is a diagram showing a schematic configuration of a video / audio coding apparatus according to conventional technique. As shown in the figure, the video encoding device according to the related art includes a video camera 2701, an audio capture unit 2702, an audio encoding unit 2703, a video capture unit 2704, and a video encoding unit 2705. As shown in the figure, the video / audio coding apparatus outputs coded audio information and coded video information as device outputs, which are transmitted or recorded as necessary.

In the figure, a video camera 2701 is
It takes in video and audio information, and separates and outputs analog audio information and analog video information. Voice capture unit 270
2 inputs analog audio information output from the video camera 2701 and outputs it as digital original audio information composed of discrete digital data. Audio encoding unit 2703
Performs compression encoding on the original audio information and outputs encoded audio information. The video capture unit 2704 receives the analog video information output from the video camera 2701 and outputs digital original video information composed of discrete digital data and composed of a plurality of still images per unit time. The video encoding unit 2705 includes the video capture unit 27
The original video information output from the input unit 04 is input, compression-encoded, and coded video information is output.

The operation of the conventional video / audio coding apparatus having the above-described configuration at the time of video / audio coding will be described below. First, the video camera 2701 takes in the video / audio information, and outputs it separately into analog audio information and analog video information.

The analog audio information is sent to the audio capture unit 2
The audio input unit 702 receives the audio data, and the audio capture unit 2702 creates digital original audio information by analog / digital conversion processing, and outputs the digital original audio information to the audio encoding unit 2703.
On the other hand, the analog video information is stored in the video capture unit 2704.
, And the video capture unit 2704
By digital conversion processing, digital original video information composed of a plurality of pieces of still image information is created, and this is output to the video encoding unit 2705.

The audio encoding unit 2703 performs an encoding process on the original audio information and outputs encoded audio information.
On the other hand, the video encoding unit 2705 performs an encoding process on the original video information and outputs encoded video information.

While video and audio are continuously captured from the video camera 2701, digitization and encoding by the audio capture unit 2702, audio encoding unit 2703, video capture unit 2704, and video encoding unit 2705 are continued. After the audio capture is completed, the digitization and encoding are also completed.

[0046]

As shown in the examples A to C of the prior art, the video coding apparatus, the audio coding apparatus, and the video / audio coding apparatus according to the conventional techniques are used for video, audio, , Or with the incorporation of video and audio, performs encoding processing and outputs coded video data, coded audio data, or coded video data and coded audio data for recording and transmission. To offer.

A. Problems of Video Coding by Conventional Technology However, a video coding device capable of real-time processing shown in A of the prior art is replaced with software for performing coding processing in a general-purpose computer system such as a personal computer (PC). If the software is intended to be executed, the software can be executed by hardware of various performances in various environments (peripheral devices, network environment, etc.). Problems.

For example, as a realization of the above-mentioned real-time video encoding apparatus as application software operating on a PC, an input video is processed in real time in accordance with the import, and the resolution of 320 × 240 is set to the MPEG1 standard. In the case of encoding according to the following, select a pattern that repeats in the order of “I” for intra-frame encoding and “P” and “B” for inter-frame predictive encoding in the order of “IBBPBB” as the encoding type. Suppose you did. In this case, if the basic hardware performance is relatively high, for example, if a control device (CPU) having an operation frequency of 166 MHz is provided, and if the software processing is performed, “IBBPB” is set according to the above setting.
It is assumed that it takes 6/30 seconds to process six frame images in the encoding type having the pattern “B”. In this case, as a result, the video can be encoded in real time at 30 frames / sec.

On the other hand, when the basic hardware performance is low, for example, when a control device (processor, CPU) having an operation frequency of 100 MHz is provided, and the software processing is performed, the above-described encoding processing is performed. 6/3
It cannot be performed in 0 seconds, and the frame rate of the obtained encoded data becomes small. In the encoding result,
If the frame rate is 30 (frames / second) or less, the video obtained by the reproduction from the encoded result has awkward motion, and in such a case, good encoding cannot be performed. turn into.

In a similar situation, for example, when such a software process is executed as one task (work) on a multitasking operating system, another application software such as a word processor is executed as another task (work). When executed, interrupted by an interrupt, or the like, it can occur even in a relatively high-performance hardware environment.

Further, the same encoding is performed with a resolution of “320 × 2
If the resolution is "640", it can be executed without any problem.
If the processing is executed at "× 400", the processing speed may not be sufficient, and a problem may occur due to a decrease in the frame rate.

The above is a problem in a case where the hardware performance is insufficient. On the contrary, a situation may occur in which the high performance hardware cannot be utilized. For example, in the case where hardware having a control device (CPU) having an operation frequency of 166 MHz performs real-time processing of an input video along with import and performs encoding according to the MPEG1 standard at a resolution of 320 × 240. In addition, if processing is performed using only “I” which is intra-frame encoding as the encoding type, if processing is performed under this condition, one-frame image is divided by １ in “I” type.
It can be processed in 0 seconds, resulting in 30 frames /
Video can be encoded in real time in seconds.

On the other hand, when the basic hardware performance is higher, for example, when a control device (CPU) having an operation frequency of 200 MHz is provided, and the software processing is performed, the “I” type Since the one-frame image processing can be performed in less than 1/30 second, hardware performance cannot be utilized. Since the use of a high-performance control device also increases the cost, this means that the cost performance is not good as a video encoding device.

In this case, for example, if processing is performed using not only the “I” type but also the “P” or “B” coding type, coded data having a high compression rate with the same image quality can be obtained. Therefore, as described above, generating encoded data with a low compression ratio only in the “I” type as described above means that the device resources have not been utilized.

The same applies to the case where computer resources (allocation of CPU time) can be used more than expected, such as when executing on a multitasking operating system, or when "32
"160x120" which is lower resolution than "0x240"
This can also occur when encoding is performed at a resolution of.

B. Problems of conventional audio encoding According to the conventional audio encoding method performed as in the first example of B-1, in a multimedia personal computer or the like equipped with a sound board, the audio processing is performed in accordance with software processing by software processing. Real-time speech coding is possible.
However, this presupposes that a device having sufficient performance for real-time encoding accompanying speech input is used. Or by selecting a control device (processor) having Alternatively, in the case of processing with a processor that does not have sufficient performance, data is recorded as a file on the way, and the recorded data is processed. There was nothing else to do.

That is, the band division encoding process used in MPEG Audio or the like is executed by software using a CPU.
In the case of trying to perform real-time processing associated with voice input by executing in the hardware environment, is the hardware environment for executing the software, typically possible by CPU performance,
It was determined whether or not it was impossible, and for example, it was not possible to perform real-time encoding at an encoding level corresponding to the CPU performance.

Further, the speech coding apparatus configured as described above is designed to input speech at a constant rate and to perform real-time encoding processing at a constant rate according to the initial settings. However, in the case of a general-purpose personal computer or the like, the processing capacity of the CPU is allocated due to the influence of other tasks due to the multitasking process, the occurrence of other interrupts, etc., and in such a state, the audio encoding process cannot be performed according to the initial setting. And it is difficult to respond to this.

Next, in the band division encoding process in which psychoacoustic analysis is performed as shown in the second example of B-2, bits are allocated according to human auditory characteristics as described above. Encoded data with good reproduction sound quality can be obtained.

However, the processing load of the division into a large number of bands and the conversion processing and comparison processing of the divided signals is large, and by performing such a psychological auditory analysis, the processing is generally reduced to about twice. The burden will increase. Therefore, at the level of a standard personal computer, it is difficult to perform real-time processing associated with voice capture when incorporating psychoacoustic analysis, and it is difficult to rely on the addition of high-performance hardware such as a dedicated processor or board. ,
After giving up the real-time processing and recording it as a file, it was necessary to take the time to perform the encoding processing.

C. Problems of Video / Audio Coding by Conventional Techniques As described above, the conventional video / audio coding apparatus described above uses, when coding audio and video, original audio information (digital audio information) and original video information (digital video information). Digital video information) is directly input to the corresponding encoding units, and is subjected to encoding processing. Therefore, the audio encoding unit and the video encoding unit are required to have a capability of reliably processing input original audio information and original video information in accordance with, for example, the MPEG standard. For example, if the sampling frequency is 48 KHz and the audio information of 1 byte per sample is input, the audio encoding unit needs to have a capability of reliably encoding the audio information of 48 Kbytes per second. In addition, the video encoding unit needs to have a capability of reliably encoding 4.6 Mbytes of video information per second if video information of 320 pixels horizontally, 240 pixels vertically, 1 pixel 2 bytes, and 30 fps is input. Becomes

For this reason, conventionally, the audio encoding unit and the video encoding unit operate independently of each other, and realize video / audio encoding by using dedicated hardware capable of guaranteeing the encoding process. Was. On the other hand, it is extremely difficult to realize an audio and video encoding unit as a software program that operates on a multitasking operating system using a general-purpose CPU without using dedicated hardware.

This is because, on a multitasking operating system, each encoding unit operates as a task, and similarly, when other software (a resident program for performing communication processing, etc.) also functions as a task. In some cases, the other task deprives the CPU of the operation time for a certain period. Since the encoding process stops during that period, it cannot always be guaranteed that the encoding software can process audio or video sufficiently. Therefore, it is not always possible to always obtain a good encoding result without causing a reproduction trouble such as interruption of audio or video.

In the case where video and audio are to be processed, there are problems other than the other tasks. That is, on a multitasking operating system, video encoding and audio encoding are usually processed as separate tasks, and thus affect each other as the other tasks described above. For example, a video cannot be expected to be uniform, and a still image constituting input video information changes every moment. Since a certain part of a series of images is very difficult to encode (compress), it may be time-consuming to process the part. In this case, even if no other software is running on the operating system, a large amount of CPU time is taken by the video encoding unit and the processing of the audio encoding unit is delayed, so that the audio is interrupted. It may happen that only a certain encoding result is obtained.

Another problem is that, when an attempt is made to configure a video / audio coding apparatus by executing software on a general-purpose computer, as in the case of A or B, the software is a computer system having various hardware capabilities. Due to what could be performed on it. for that reason,
Apart from the problem mentioned above, that is, the problem that the computational power assigned to the encoding is reduced in a certain period in the case where it is possible to encode both audio and video on average, If the hardware on which the software is executed does not have sufficient computational capabilities in the first place, video and audio encoding may not be performed with the initial settings at the time of software design. There is. In such a case, unless the computing function consumed by video encoding is reduced promptly in accordance with the operating computer system, a good encoding result cannot be obtained, and a situation in which audio is interrupted during reproduction is caused. .

Of course, due to the effect of the audio encoding process,
Although there is a possibility that the encoding result of the video encoding may have a problem, in general, when comparing the video and audio corresponding to the same time, the video has a larger amount of data and also has a larger data amount than the lack of the video data. Since the effect of the lack of audio data has a large effect at the time of reproduction, it can be generally said that the problem in audio coding has a higher specific gravity, and that there is a greater demand for prevention of audio interruption. .

As described above, when the encoding software is executed on a general-purpose computer such as a personal computer from A to C, it is intended to perform real-time encoding processing of video, audio, or video / audio along with the import. Can be said to have the following problems.

(1) The effect of the performance of the hardware executing the software is large. If the hardware performance is low, good coding results cannot be obtained, and if the hardware performance is high, there is a possibility that the device resources cannot be used. (2) When the software is executed on a multitasking operating system, the influence of other tasks is large. The degree of occupation of device resources by other tasks depends on
This has substantially the same effect as the level of hardware performance in (1). (3) Furthermore, when video and audio are to be processed, it may happen that video encoding and audio encoding affect each other as other tasks.

The present invention has been made in view of the above circumstances, and in a coding method for performing a video coding process in real time upon capturing a video, a basic computer for executing the coding method is used. It is an object of the present invention to provide a video coding method capable of appropriately setting a coding parameter including a resolution and a coding type according to a capability and utilizing a device resource to obtain a good coding result. Aim.

The present invention also relates to an encoding method for performing a video encoding process in real time upon capturing a video. By properly setting the encoding parameters including the encoding type and utilizing the device resources,
It is an object of the present invention to provide a video coding method capable of obtaining a good coding result.

The present invention also relates to an encoding method for performing real-time audio encoding processing in response to audio capture, wherein the encoding processing is performed in accordance with the basic capability of a computer that executes the encoding method. And to provide a speech encoding method capable of obtaining good encoding results by utilizing device resources.

Further, the present invention relates to a coding method for performing a real-time voice coding process along with a voice capture, wherein the coding is performed in accordance with the current capability of a computer which executes the coding method. An object of the present invention is to provide a speech encoding method capable of controlling a process and utilizing a device resource to obtain a good encoding result.

Further, the present invention relates to an encoding method for performing a real-time audio encoding process in response to a speech capture, wherein a psychoacoustic analysis is performed in accordance with the basic ability of a computer which executes the encoding method. It is an object of the present invention to provide a speech encoding method capable of executing the alternative processing of (1) and utilizing a device resource to obtain a good encoding result.

The present invention also relates to an encoding method for performing real-time video encoding and audio encoding processing along with video and audio capture, which corresponds to the basic capability of a computer that executes the encoding method. It is another object of the present invention to provide a video / audio coding method capable of controlling a video coding process and utilizing a device resource to obtain a good coding result without sound interruption.

The present invention also relates to an encoding method for performing video encoding and audio encoding processing in real time accompanying video and audio capture, and to a computer capable of executing the encoding method at that time. Correspondingly, an object of the present invention is to provide an audio encoding method capable of controlling a video encoding process and utilizing a device resource to obtain a satisfactory encoding result without sound interruption.

The present invention also provides a video encoding method, an audio encoding method, and a video encoding apparatus, an audio encoding apparatus, and a video / audio encoding apparatus that execute the video / audio encoding method as described above. The purpose is to do.

Further, the present invention can be implemented on a general-purpose computer such as a personal computer to realize the above-described video encoding method, audio encoding method, and video / audio encoding method. It is an object of the present invention to provide an encoding program and a recording medium recording an audiovisual encoding program.

[0078]

According to a first aspect of the present invention, there is provided a video encoding method for encoding a video, wherein the video encoding method comprises the steps of: For the original video information consisting of
A video encoding step of encoding one or more of the still image information according to encoding parameters described below;
The resolution of the original video information, the frame rate required when reproducing encoded data obtained by encoding,
One of a processing performance indicating a processing capability of an encoding device that executes the video encoding step, or one or a plurality of encoding parameters that affects a processing amount of an encoding process in the video encoding step. Based on the above, a coding parameter determining step of determining one or more coding parameters is executed.

Further, the video encoding method according to claim 2 is the video encoding method according to claim 1,
The processing capability of the encoding device that performs the video encoding step is determined, and a processing capability determining step of outputting a determination result is further performed.

According to a third aspect of the present invention, in the video encoding method according to the first or second aspect, the encoding parameter is a resolution in an encoding process performed on the original video information, an intra-frame encoding, Alternatively, it includes one or more of a coding type indicating predictive coding and a detection range for detecting a motion vector used for the predictive coding.

According to a fourth aspect of the present invention, in the video encoding method of the second aspect, in the processing capability determining step, the determination is made based on a type of a control device of the video encoding method. is there.

According to a fifth aspect of the present invention, in the video encoding method according to the second aspect, in the processing capability determining step, the determination is made based on a time required for the encoding process in the encoding step. is there.

According to a sixth aspect of the present invention, in the video encoding method according to the second aspect, in the processing capability determination step, the input original image information is temporarily stored, and when storing the original image information, A video buffering step of sequentially storing a series of still image information constituting the information, and sequentially discarding the still image information read out in the encoding step and subjected to the encoding processing; Executing a frame rate control step of controlling the storage of the series of still image information in the buffering step to be performed at a constant frame rate determined based on the given frame rate. The above determination is made based on the storage amount of the original video information temporarily stored in.

According to a seventh aspect of the present invention, there is provided a speech encoding method for encoding a speech by a band division encoding method, wherein the set frequency fs and the numerical value used for the encoding process are used. , A conversion constant n, a voice input step of inputting a voice to be encoded, and a sampling voice data generated using a sampling frequency determined based on the stored set frequency fs. And the number of sampled audio data obtained when the set frequency fs is used as the sampling frequency is m
And the number determined based on the conversion constant n is m ′
An audio data conversion step of outputting converted audio data composed of m audio data, including m ′ sampled audio data, and a band division for dividing the converted audio data into bands to obtain M band signals A coding step for allocating coded bits only to a band signal having a frequency equal to or lower than the limit frequency among the band signals, using a frequency fs / 2n obtained from the stored set frequency fs and the conversion constant n as a limit frequency. And a quantization step of performing quantization based on the allocated coded bits, a coding step of outputting the quantized data as coded data, and coded data for recording the output coded data. And a recording step.

In the voice coding method according to the present invention, the input voice sampling step may be performed by sampling the input voice using the stored set frequency fs as a sampling frequency. , M sampling audio data, and in the audio data converting step, sampling audio data is extracted from the m sampling audio data every (n−1) sampling data, and two adjacent sampling audio data are extracted. Between the extracted sampled audio data, (n-1)
Is converted into m converted audio data by inserting the audio data.

According to a ninth aspect of the present invention, in the audio encoding method according to the eighth aspect, in the audio data conversion step, converted audio data in which each of the extracted sampled audio data is continuous by n pieces is created. It is.

According to a tenth aspect of the present invention, in the speech encoding method according to the seventh aspect, in the input speech sampling step, a frequency fs / n obtained from the stored set frequency fs and the conversion constant n is sampled. As the frequency, m / n pieces of sampled audio data are created by sampling the input audio, and in the audio data conversion step, two adjacent sampled audio data are generated based on the sampled audio data. In between, (n-1) audio data is inserted and converted into m converted audio data.

In the audio encoding method according to the eleventh aspect, in the method of the tenth aspect, in the audio data conversion step, the m / n sampled audio data is converted audio data that is continuous by n pieces each. Is to create.

A speech encoding method according to a twelfth aspect of the present invention is the audio encoding method according to any one of the seventh to eleventh aspects, wherein the sampling audio data is temporarily held in an input buffer; Checking the amount of data in the buffer, comparing it with a preset value, and performing an input buffer monitoring step of changing the value of the conversion constant n stored in the register based on the result of the comparison; In the input audio sampling step, the sampled audio data is written to the input buffer. In the audio data conversion step, the sampled audio data is read out from the input buffer and converted.

A speech encoding method according to a thirteenth aspect is characterized in that, in the method of any one of the seventh to eleventh aspects, the amount of encoded data per unit time output in the encoding step is examined, and A coded data monitoring step of comparing a value set in advance and changing the value of the conversion constant n stored in the register based on the result of the comparison is executed.

According to a fourteenth aspect of the present invention, in the voice coding method for coding a voice using a band division coding method, a control method for storing a control constant used for the coding is provided. A constant storage step, a sampling step of sampling the input voice and outputting sampling data, and performing band division on the sampling data obtained in the sampling step,
A band dividing step of outputting band signal data; a coding bit allocating step of allocating coded bits to the band signal data obtained in the band dividing step; and Quantizing the signal data, a quantization step of outputting a quantization value, and an encoding step of outputting encoded data based on the quantization value obtained in the quantization step,
On the basis of the stored control constants, the band division step, the coded bit allocation step, the quantization step, and a coding processing control step for controlling data processing in the coding step are executed.

According to a fifteenth aspect of the present invention, in the method of the fourteenth aspect, in the control constant storing step, a unit period determination constant k is stored in the unit period determination constant register as the control constant. In the encoding process control step, the number of sampling data to be targeted in one band division process in the band division step is p, and the time corresponding to the p pieces of sampling data is set as a unit period. For every p pieces of sampling data to be sampled, it is determined whether the corresponding unit period is a coding target period or a non-coding target period based on the stored unit period determination constant. Only when it is determined to be the encoding target period, control is performed such that the sampling data of the unit period is output to the band division step, and When the time period is determined to the encoding target out period, in the encoding step are those wherein the determination control step of controlling to output the fixed coded data previously stored as encoded data.

In the speech coding method according to a sixteenth aspect of the present invention, in the method according to the fifteenth aspect, in the determining control step, the stored unit period determination constant k is set to an arbitrary integer by setting the i-th unit period to ti. From n, i = n × k + 1
Is satisfied, it is determined that the unit period ti is the encoding target period.

According to a seventeenth aspect of the present invention, in the speech encoding method according to the fourteenth aspect, in the control constant storing step, an arithmetic processing determination constant q is stored in the arithmetic processing determination constant register as the control constant. And the encoding process control step is included in the band division step and is based on the stored arithmetic processing determination constant q,
This is a calculation processing stop step for controlling the calculation processing in the band division step to be terminated halfway.

The speech coding method according to claim 18 is the method according to claim 17, wherein, in the step of stopping the arithmetic processing, the arithmetic processing of the basic low-pass filter in the band division step is performed at both ends of the filter. The minute is controlled so as to be interrupted halfway.

According to a nineteenth aspect of the present invention, in the speech encoding method according to the fourteenth aspect, in the control constant storing step, a band selection constant r is stored in the band selection constant register as the control constant. The encoding process control step includes the step of allocating the encoded bit and the step of performing the quantization on only the band signal data output from the band division step selected based on the stored band selection constant r. This is a band thinning step for controlling to execute the processing in the step.

A speech encoding method according to a twentieth aspect of the present invention is the audio encoding method according to the nineteenth aspect, wherein in said band thinning step, the stored band signals are output from the M band signal data outputs obtained in the band division step. R is the selection constant
The band signal data is selected every other.

A speech encoding method according to claim 21 is the method according to any one of claims 14 to 20, wherein
A processing status monitoring step of acquiring a status of data processing in voice encoding and changing a value of the stored control constant according to the obtained status is executed.

According to a twenty-second aspect of the present invention, in the audio encoding method according to the twenty-first aspect, in the processing status monitoring step, an audio buffering step of temporarily storing sampling data in an input buffer; The amount of data to be compared with a preset value,
And an input monitoring step of changing the control constant based on the result of the comparison.

According to a twenty-third aspect of the present invention, in the speech encoding method according to the twenty-first aspect, the processing status monitoring step comprises: determining an amount of the encoded data output per unit time in the encoding step. This is an encoding monitoring step of comparing with a preset value and changing the value of the control constant based on the result of the comparison.

According to a twenty-fourth aspect of the present invention, in the audio encoding method for encoding original audio information obtained by digitizing audio using a band division encoding method, the input audio Sampling processing, a sampling step of outputting sampling data, performing band division on the sampling data obtained in the sampling step, and outputting a band signal data; and a band division step of outputting band signal data. For the band signal data, a coded bit allocation step of allocating coded bits, a bit allocation control step of controlling the allocation in the coded bit allocation step by a psychological auditory analysis alternative control method, According to the allocation, the band signal data is quantized, A quantization step of outputting the reduction value,
And an encoding step of outputting encoded data based on the quantization value obtained in the quantization step.

A speech coding method according to a twenty-fifth aspect of the present invention is the method of the twenty-fourth aspect, wherein the bit allocation control step is performed based on the psychological auditory analysis alternative control for the band signal data obtained in the band division step. This is a sequential bit allocation step for controlling to perform coded bit allocation in accordance with a bit allocation order predetermined by the system.

A speech coding method according to a twenty-sixth aspect is characterized in that, in the method of the twenty-fourth aspect, the bit allocation control step is a method of substituting a psychoacoustic analysis for band signal data obtained in the band division step. This is a band output adaptive bit allocation step for controlling to perform coded bit allocation based on weighting of each band determined in advance by a system and an output level of each band signal data.

According to a twenty-seventh aspect of the present invention, in the speech encoding method according to the twenty-fourth aspect, the bit allocation control step includes a step of substituting a psychoacoustic analysis for the band signal data obtained in the band division step. Improved band output adaptation for controlling to perform coding bit allocation based on weighting to each band predetermined by the method, weighting to the number of bit allocations for each band, and output level of each band signal data This is a bit allocation step.

[0105] In the speech coding method according to claim 28, in the method according to claim 24, the bit allocation control step is performed for each band signal data with respect to the band signal data obtained in the band division step. The minimum audible limit is compared with the minimum audible limit, and is controlled so that bit allocation is not performed on band signal data determined to be less than the minimum audible limit by the above comparison, and bit allocation for other bands is increased. This is a comparison step.

Further, in the video / audio encoding method according to claim 29, when encoding video and audio, a part or all of the processing steps included in the two encoding processes is performed.
In a video / audio coding method executed using a common computer resource, video / audio information composed of original video information composed of a plurality of still image information representing still images per unit time and original audio information representing audio. Is input, an audio buffering step of temporarily accumulating the original audio information, and reading the original audio information accumulated in the audio buffering step, encoding the read original audio information, Using the audio encoding step of outputting the encoded audio information and the encoding load reference information indicating the degree of the encoding load of the video, the processing capability of the video / audio encoding process is determined, and based on the result of the determination, A coding load evaluation step of controlling coding of original video information in a video coding step described later; and According kick control, the still picture information constituting the original video information input to coding processing, and executes a video encoding step of outputting the coded video information.

[0107] In the video / audio coding method according to claim 30, in the method according to claim 29, the encoding load evaluation step is performed when the still image information constituting the original video information is input. The total amount of the original audio information stored in the buffering step and the coding load evaluation information are obtained based on the coding load reference information, and the coding load evaluation information is compared with a preset load limit, Still image information is output when the encoding load evaluation information has not reached the load limit, and the still image information is discarded when the encoding load evaluation information has reached the load limit.

A video / audio coding method according to claim 31 is the method according to claim 29, wherein analog video information is input and video resolution information described later is output.
The analog video information is converted into original video information composed of a plurality of discrete digital pixel information and a plurality of still image information having a resolution according to the video resolution information, and output so as to be processed in the video encoding step. In the encoding load evaluation step, the total amount of the original audio information accumulated in the audio buffering step and encoding load reference information indicating the degree of the image encoding load are calculated. Calculating coding load evaluation information based on the coding load evaluation information, obtaining video resolution information representing a resolution of a video used for video coding, based on the coding load evaluation information, and outputting the video resolution information; In the converting step, when the video resolution information is output, the still image information is matched with the video resolution information in accordance with the video resolution information. Performs coding processing Te, and outputs the coded video information.

In the video / audio coding method according to claim 32, in the method according to claim 29, in the coding load evaluation step, the coding load evaluation information is output so as to be processed in the video coding step. Things,
In the video coding step, the still image information is subjected to a coding process by a processing amount calculated using the output coding load evaluation information, and is output as coded video information.

A video and audio encoding method according to claim 33 is the method according to any one of claims 29 to 31, wherein in the audio encoding step, the original audio information accumulated in the audio buffering step is read. Calculating the total amount of the read original audio information and outputting it as a processed audio information amount, and thereafter, encoding the original audio information and outputting it as encoded audio information. In the evaluation step, an original audio input amount is obtained based on the elapsed time and the input amount of the original audio information per time, and a predicted audio buffer amount which is a difference between the original audio input amount and the processed audio information amount is calculated. Then, the coding load evaluation information is obtained using the predicted audio buffer amount.

In the video / audio coding method according to claim 34, in the method according to any one of claims 29 to 31, in the coding load evaluation step, when the still image information is input, an elapsed time and Determining the original audio input amount based on the input amount per time of the original audio information, and obtaining the processed audio information amount based on the total amount of the encoded audio information output in the audio encoding step,
Further, after calculating a predicted audio buffer amount which is a difference between the obtained original audio input amount and the processed audio information amount, the coding load evaluation information is obtained using the predicted audio buffer amount. It is.

The video / audio coding method according to claim 35 is the method according to any one of claims 29 to 31, wherein a change in the result of the determination in the coding load evaluation step is monitored, and Correspondingly, the coding load reference information is set.

A video coding apparatus according to claim 36 is a video coding apparatus for coding a video, wherein the video coding is performed on the original video information consisting of a plurality of pieces of still image information obtained by digitizing the video. One or more of the image information
Video encoding means for encoding according to encoding parameters to be described later, one or more resolutions as one encoding parameter, and intra-frame encoding, forward prediction encoding, backward prediction encoding, and bidirectional prediction encoding One or more encoding types among the encoding types including each type of encoding are used as other encoding parameters, and the encoding parameters for determining the processing amount of the encoding means are set to a given frame rate. Encoding parameter determining means for determining based on the coding parameter.

A speech encoding apparatus according to claim 37 is a speech encoding apparatus for encoding a speech by a band division encoding method, wherein the set frequency fs and the numerical value used for the encoding process are used. , A conversion constant n, audio input means for inputting audio to be encoded, and sampling frequency determined based on the stored set frequency fs, thereby generating sampled audio data. The input audio sampling means and the number of sampled audio data obtained when the set frequency fs is used as the sampling frequency is m, and the number determined based on the conversion constant n is m ′, and m ′
Audio data conversion means for outputting converted audio data consisting of m audio data, including a number of sampled audio data; band dividing means for dividing the converted audio data into bands to obtain M band signals; Set frequency fs stored
A coding frequency allocating means for allocating coded bits only to a band signal having a frequency equal to or lower than the limited frequency, with the frequency fs / 2n obtained from the conversion constant n and the limited frequency, And a coded data recording means for recording the output coded data. .

A speech coding apparatus according to a thirty-eighth aspect of the present invention is a speech coding apparatus for performing coding on a voice using a band division coding method, wherein the control for storing a control constant used for the coding is performed. Constant storage means, sampling processing of input audio, sampling means for outputting sampling data, band division means for performing band division on the sampling data obtained by the sampling means, and outputting band signal data, Coding bit allocating means for allocating coded bits to the band signal data obtained by the band dividing means, and quantizing the band signal data in accordance with the coded bit allocation; And encoding means for outputting encoded data based on the quantized value obtained by the quantization means. Based on the stored control constant is obtained by a coding process control means for controlling the data processing in the band dividing means, the coding bit allocation means, the quantization means, and said encoding means.

A speech coding apparatus according to a thirty-ninth aspect of the present invention is a speech coding apparatus for performing coding on a speech by using a band division coding method. Sampling means for outputting, band division means for performing band division on the sampling data obtained by the sampling means, and outputting band signal data; and code conversion for the band signal data obtained by the band division means. Coded bit allocating means for allocating coded bits, bit allocation control means for controlling allocation in the coded bit allocating means by a psychological auditory analysis alternative control method, and allocating the coded bits to the band signal data. Quantizing means for performing quantization and outputting a quantized value, and an amount obtained by the quantizing means Based on reduction value is obtained by a coding means for outputting coded data.

The video / audio encoding apparatus according to claim 40, when encoding video and audio, performs a part or all of the processing steps included in the two encoding processes.
In a video / audio coding apparatus executed using a common computer resource, video / audio information composed of original video information composed of a plurality of still image information representing still images per unit time and original audio information representing audio. Is input, audio buffering means for temporarily storing the original audio information,
Audio encoding means for reading the original audio information stored in the audio buffering means, encoding the read original audio information, and outputting encoded audio information, and a code indicating the degree of load of video encoding Using the coding load reference information, determine the processing capability of the video and audio encoding device,
Based on the result of the determination, an encoding load estimating means for controlling the output of the original video information to an image encoding means to be described later; And video encoding means for encoding the still image information and outputting encoded video information when the still image information to be input is input.

A video encoding program recording medium according to claim 41 is a recording medium on which a video encoding program for encoding a video is recorded, comprising a plurality of still image information in which the video is digitized. For the video information, a video encoding step of encoding one or more of the still image information according to encoding parameters described below, and one or more resolutions as one encoding parameter,
One or more encoding types among encoding types including intra-frame encoding, forward predictive encoding, backward predictive encoding, and bidirectional predictive encoding are used as the other encoding parameters. And a coding program for executing a coding parameter determination step of determining a coding parameter for determining a processing amount of the coding step based on a given frame rate.

[0119] Further, the audio encoding program recording medium according to claim 42 is a recording medium in which an audio encoding program for encoding audio by a band division encoding method is recorded. A storage step of storing a set frequency fs and a conversion constant n, a voice input step of inputting a voice to be encoded, and a sampling frequency determined based on the stored set frequency fs. The input voice sampling step of creating the sampled voice data, and the number of sampled voice data obtained when the set frequency fs is used as the sampling frequency is m,
An audio data conversion step of outputting converted audio data composed of m audio data, including m ′ sampled audio data, where m ′ is a number determined based on the conversion constant n, where ≧ m ′; The converted audio data is
A band division step of dividing the band to obtain M band signals, and a frequency fs / 2n obtained from the stored set frequency fs and the conversion constant n as a limit frequency. A coded bit allocation step of allocating coded bits only to a signal, a quantization step of performing quantization based on the allocated coded bits, and a coding step of outputting the quantized data as coded data, And a coded data recording step of recording the output coded data.

[0120] Further, the audio encoding program recording medium according to claim 43 is a recording medium in which an audio encoding program for encoding audio by using a band division encoding system is recorded. A control constant storing step of storing a control constant to be used; a sampling step of sampling an input voice to output sampling data; and performing band division on the sampling data obtained in the sampling step, Outputting a band division step, allocating coded bits to the band signal data obtained in the band division step, and quantifying the band signal data according to the coded bit allocation. A quantization step of performing quantization and outputting a quantization value. Based on the quantized values obtained by the flop,
An encoding step of outputting encoded data; and an encoding step of controlling data processing in the band division step, the encoded bit allocation step, the quantization step, and the encoding step based on the stored control constant. It records an encoding program for executing the processing control steps.

Further, according to a sound encoding program recording medium of the present invention, an input sound is sampled on a recording medium on which an audio encoding program for encoding a sound by using a band division coding method is recorded. Process,
Sampling step of outputting sampling data, performing band division on the sampling data obtained in the sampling step, outputting a band signal data, and performing band division on the band signal data obtained in the band division step. A coded bit allocation step of allocating coded bits; a bit allocation control step of controlling the allocation in the coded bit allocation step by a psychological auditory analysis alternative control method; and An encoding program that performs quantization of signal data and outputs a quantization value, and an encoding step of outputting encoded data based on the quantization value obtained in the quantization step, It is recorded.

Further, in the video / audio coding program recording medium according to claim 45, in coding video and audio, a part or all of the processing steps included in the two coding processes may be performed by a common computer. In a recording medium on which a video / audio encoding program to be executed using resources is recorded, a video composed of original video information composed of a plurality of still image information representing still images per unit time and original audio information representing audio. When audio information is input, an audio buffering step of temporarily storing the original audio information, and reading the original audio information stored in the audio buffering step, and encoding the read original audio information Then, using the audio encoding step of outputting encoded audio information and the encoding load reference information indicating the degree of the encoding load of the video, An encoding load evaluating step of controlling encoding of original video information in a video encoding step to be described later based on a result of the determination, and a control in the encoding load evaluating step; And a video encoding step of encoding the input still image information constituting the original video information and outputting the encoded video information.

[0123]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. The video encoding method according to the first embodiment of the present invention determines a certain parameter among a plurality of encoding parameters, and determines another parameter based on a set frame rate and the determined parameter. .

FIG. 1 is a block diagram showing a configuration of a video encoding apparatus according to Embodiment 1 of the present invention. As shown in the figure, the video encoding apparatus according to the first embodiment includes an encoding unit 101 and an encoding parameter determination unit 102. The encoding unit 101 includes a DCT processing unit 103, a quantization unit 104, variable length encoding means 105,
Bit stream generation means 106, inverse quantization means 10
7, the inverse DCT processing unit 108 and the predicted image generation unit 109, and the encoding parameter determination unit 102 includes a resolution reference table 110.

The encoding means 101 inputs video data consisting of a series of still images in which video has been digitized, as input image data, performs an encoding process according to the set encoding parameters, and outputs encoded data. . Each still image data constituting the input image data is called a frame image. The encoding parameter is provided from an encoding parameter determination unit 102 described later, and includes a parameter indicating an encoding type and a parameter indicating a resolution. The parameter indicating the encoding type is
This indicates intra-frame encoding processing or forward prediction encoding processing, and the encoding unit 101 performs intra-frame encoding or forward prediction encoding according to the parameter. The parameter indicating the resolution is input to a DCT processing unit described later, and the encoding process is performed at the resolution.

In the encoding means 101, first, the DCT means 103 performs DCT (discrete cosine transform) processing on the input image data to output DCT transformed data, and then the quantization means 104 The quantized data is output by performing a quantization process on the transformed data, and then the variable-length encoding unit 105 performs a variable-length encoding process on the quantized data, so that the compressed encoded variable-length Encoded data is created. The variable length coded data is input to the bit stream generating means 106, and the bit stream generating means 106 outputs coded data, which is an output from the device, as a bit stream that can be transmitted or recorded.

The inverse quantization means 107 is provided for the quantization means 104
The inverse DCT unit 108 performs inverse quantization on the quantized data output from the inverse quantization process, which is an inverse process of the quantization process, and outputs the inversely quantized data. Perform inverse DCT processing, which is the reverse of the processing,
The inverse DCT transform data is output, and the inverse DCT transform data is input to the predicted image generation means 103 and output as predicted image data. According to the encoding parameters,
When the inter-frame encoding process using the predicted image is performed, the difference data between the predicted image data and the input image data is input to the DCT unit 103, so that the encoding unit 101 Will be performed.

In the video coding apparatus according to the first embodiment, the coding parameter determining means 102 determines the resolution from the designated frame rate and coding pattern using the included resolution reference table 110. , And outputs the above-mentioned encoding parameter including the parameter indicating the determined resolution to the encoding means 101.

The video encoding apparatus according to the first embodiment is realized by executing a video encoding program under the control of a processing control device (CPU) in a personal computer (PC). In execution of the processing, it is assumed that the following five conditions are satisfied.

(1) It is assumed that both the intra-frame encoding process and the forward prediction encoding process require a time proportional to the resolution of the frame image to be processed. (2) The processing time when performing forward prediction encoding is
It is assumed that it takes six times as long as the case where the intra-frame encoding is performed. (3) When intra-frame encoding is performed, the obtained encoded data amount is 1/10 of the input image data. When forward prediction encoding is executed, the obtained encoded data amount is the input image data. It is assumed that the amount is 1/60 of the image data. (4) The CPU of the PC that realizes this device has an operating frequency of 10
When operating at 0 MHz, when a frame image having a resolution of 320 × 240 is encoded using intra-frame encoding, it can be processed in 1/24 second. (5) The processing capacity of this device is based on the CPU mounted on this device.
Is proportional to the operating frequency of That is, the processing time of the encoding process in the present apparatus is proportional to the reciprocal of the operating frequency. Here, the operating frequency of the CPU mounted on this device is 10
0 MHz, the frame rate specified at the start of encoding is 24 frames / sec, and the encoding patterns as a combination of encoding types are pattern 1 “II”, which is a pattern in which all are only “I”, and two frames. It is assumed that there are two types, a pattern 2 “IP” that repeats “I” and “P” every time. Here, the intra-frame encoding is represented by “I” and the forward prediction encoding is represented by “P”.

The operation of the video encoding apparatus according to the first embodiment configured as described above under the above settings will be described below. First, the video to be encoded is digitized and input to the encoding unit 101 of the encoding device as a series of frame images. FIG. 2 is a flowchart illustrating the operation of the encoding unit 101. The operation of the encoding means 101 will be described below with reference to FIG. Note that the encoding parameter determination unit 102 always instructs the encoding unit 101 to perform intra-frame encoding for the first frame image at the start of encoding.

In step A01, the encoding parameters inputted from the encoding parameter determining means 102 are judged. If intra-frame encoding is instructed, the processing in step A02 and subsequent steps is executed, and the forward prediction code is executed. If the conversion has been instructed, the processing after step A07 is executed.

When step A02 and subsequent steps are executed,
It looks like this: In step A02, DCT processing means 10
3 divides the input frame image into blocks of 8 pixels × 8 pixels based on the resolution instructed by the encoding parameter determination unit 102, performs two-dimensional discrete cosine transform for each of the divided blocks, and converts the DCT transformed data. Output. Next, in step A03, the quantization means 104
A quantization process is performed on the T-transformed data using a predetermined value, and the quantized data is output. Then, in step A04, the variable-length coding unit 105 performs variable-length coding on the quantized data and outputs variable-length coded data.
In step A05, the bit stream generation unit 1
06 is a device output of the video encoding device using the variable length encoded data output from the variable length encoding unit 105, the resolution output from the encoding parameter determination unit 102, and the encoding type. Generates and outputs a bit stream as an encoding result.

In step A06, it is determined whether or not the encoding has been completed. If it is determined that the encoding has been completed, the process ends. On the other hand, if the encoding is not completed, the process returns to step A01, and the steps after step A01 are executed.

On the other hand, when step A07 and subsequent steps are executed according to the judgment in step A01, the following is performed. First, in step A07, the inverse quantization means 107
The quantization means 104 inversely quantizes the quantized data already output for the immediately preceding frame image, and outputs the inversely quantized data. Next, in step A08, the inverse DCT processing unit 108 performs a two-dimensional inverse cosine transform, which is an inverse process of the two-dimensional discrete cosine transform, on the inverse-quantized data for each block of 8 × 8 pixels divided by the DCT processing unit 103. Performs a discrete cosine transform and outputs inverse DCT transform data. In step A09, the predicted image generation unit 109
Generates and outputs a predicted image based on the inverse DCT transform data.

In step A10, DCT processing means 103
Is the input frame image and the predicted image generation unit 109
Is divided into blocks of 8 pixels × 8 pixels based on the respective indicated resolutions, and for each of the divided blocks, the data of the predicted image is subtracted from the data of the input frame image to obtain a difference. Get the data. Then, two-dimensional discrete cosine transform is performed on the difference data for each divided block, and DCT transform data is output. Steps A11 to A14 after the DCT transform data are output are executed in the same manner as steps A03 to A06.

As described above, the encoding means 101 determines, for each input frame image, the steps A02 to A06 or the steps A07 to A1 based on the determination in the step A01.
Four processes will be performed. Step A02-A
06 is an intra-frame encoding, and steps A07 to A1
Reference numeral 4 denotes forward coding processing based on a predicted image using the coding result for the immediately preceding frame image, and this switching is performed according to the input coding parameter in the determination of step A01. Things.

(Table 1) shows the encoding parameter determining means 10
2 is a table showing a resolution reference table 110 included therein. FIG. 3 is a flowchart showing the operation of the encoding parameter determining means 102. Hereinafter, the operation of the encoding parameter determining unit 102 that determines the encoding parameter and outputs the encoding parameter to the encoding unit 101 will be described with reference to Table 1 and in accordance with the flow of FIG.

[0139]

[Table 1]

The resolution reference table 110 shown in (Table 1)
Are created in advance of encoding. The table can be created, for example, based on empirical knowledge or using results of experimental coding, simulation, or the like, in consideration of the conditions described below.
The “input” column in (Table 1) shows the set frame rate and the designated parameter, and the “output” column shows the parameter determined in response to the input. As shown in the table, in the first embodiment, the resolution is determined according to the frame rate and the coding pattern. The frame rate is fixedly set to "24 (frames / second)", while the coding pattern is indicated by "II" pattern 1 and "IP" pattern 2. Can be. Of these, pattern 1
"II" means that intra-frame encoding (I) is performed on all frame images, and pattern 2 "IP"
Means that intra-frame coding (I) and forward prediction coding (P) are repeated every two frames.

The creation of the reference table is performed in consideration of the following conditions. First, the forward prediction encoding process requires a larger processing amount than the intra-frame encoding process because the processes performed by the inverse quantization unit 107, the inverse DCT processing unit 108, and the predicted image generation unit 109 are added. Second, when the input image is coded at a high resolution, the processing amount is larger than when the coding is performed at a low resolution. In consideration of these conditions, the resolution reference table 110 is created so that the encoding process can be executed at the highest possible resolution while realizing the specified frame rate.

First, in step B01 of the flow of FIG. 3, the coding parameter determining means 102 obtains the resolution reference table 110 from the specified frame rate of 24 frames / sec and the coding pattern (II or IP). , The resolution of the frame image to be encoded is determined.

Next, in step B02, the coding parameter determining means 102 instructs the coding means 101 on the resolution determined in step B01 and performs processing on the processing object so that the specified coding pattern can be realized. Indicate the coding type (I or P) to be used for a certain frame image.

Thereafter, in step B03, it is determined whether or not the coding has been completed. If it is determined that the coding has been completed, the process is completed. On the other hand, if not finished, the process returns to step B02, whereby the output of the encoding parameters to the encoding means 101 is repeated.

Coding is performed by the above-described operations of the coding means 101 and the coding parameter determination means 102. Table 2 shows the coding in the video coding apparatus according to the first embodiment. It is a table | surface which shows the result of having performed conversion.

[0146]

[Table 2]

Table 2 shows the resolutions (determined parameters) determined by the encoding apparatus according to the first embodiment and the codes using those parameters for the two specified encoding conditions. And the frame rate (encoding result) obtained as a result of the encoding process. For the numerical values of the encoding results shown in (Table 2), the encoding pattern "I
I ", if the resolution is 320 × 240,
The frame rate in other cases is calculated based on the fact that the frame rate can be processed at 4 frames / sec. When the coding pattern is IP and the resolution is 160 × 120, the frame rate requires six times as long as the processing of P and the processing of P when the resolution is 1/4. From what we can do, we need to encode two frame images (1
/ 24 + 6/24) ÷ 4 = 0.073 seconds can be calculated, from which it can be calculated to be 27.428 frames / second.

For comparison, Table 3 shows the operation results when encoding is performed using a conventional video encoding device.

[0149]

[Table 3]

In Table 3, the same calculation as in Table 2 is performed, and in coding pattern II,
The frame rate in other cases is calculated based on processing at 24 frames / sec when the resolution is 320 × 240.

In a video coding apparatus according to the prior art, the frame rate obtained as a result of coding is not taken into consideration,
The encoding type (pattern) or the resolution has been determined. Therefore, it is difficult to set the frame rate obtained as a result of the encoding process close to a desired value, and there is a case where a setting that results in an unnecessary value must be selected. Was. On the other hand, in the video encoding apparatus according to the first embodiment, the resolution is determined according to the specified encoding type (pattern) in consideration of the frame rate as the encoding result. As shown in the comparison between Table 3 and Table 3, it can be seen that encoding at a higher resolution is executed while realizing a frame rate close to the designated frame rate.

As described above, according to the video encoding apparatus of the first embodiment, the encoding means 101 and the encoding parameter determination means 10 including the resolution reference table 110 are included.
2, the coding parameter determining means 102
Determines the resolution corresponding to the specified frame rate and encoding type, and outputs the encoding parameter to the encoding means 101. The encoding means 101 performs the encoding process in accordance with the encoding parameter. , It is possible to perform encoding at a higher resolution while realizing the required conditions.

In the video coding apparatus according to the first embodiment, the resolution is determined in accordance with the specified coding pattern. However, by using the same reference table, the resolution can be set to the specified resolution. It is also possible to determine the coding pattern (type) correspondingly, and it is possible to perform a process of obtaining a coding result with a higher compression rate under the required frame rate and coding pattern. Become.

Embodiment 2 The video encoding method according to the second embodiment of the present invention determines encoding parameters based on a set frame rate in accordance with the processing capability of the encoding device. The processing capability is determined based on the operating frequency.

FIG. 4 is a block diagram showing a configuration of a video encoding device according to Embodiment 2 of the present invention. As shown in the figure, the video encoding device according to the second embodiment includes an encoding unit 201, an encoding parameter determination unit 202, and a processing capability determination unit 211. Processing means 203, quantization means 20
4. Variable length coding means 205, bit stream generation means 206, inverse quantization means 207, inverse DCT processing means 20
8 and the predicted image generation means 209, and the coding parameter determination means 202 includes a coding pattern reference table 210.

The coding means 201 is the same as the coding means 101 of the video coding apparatus according to the first embodiment. For images,
At the indicated resolution and intra-frame encoding (I),
Alternatively, encoding is performed using a designated encoding type such as forward prediction encoding (P).

The encoding parameter determining means 202 determines an encoding parameter according to the result of the judgment by the processing capability judging means 211 and outputs it to the encoding means 201. The processing capability determining unit 211 determines the encoding processing capability of the encoding device, and outputs a result of the determination to the encoding parameter determining unit 202. In the second embodiment, the processing capacity determination unit 211
Outputs the “operating frequency of the CPU” indicating the processing capability of the encoding apparatus as a determination result. The encoding parameter determination unit 202 outputs the specified frame rate and resolution, and the operation as the determination result. A coding pattern is determined from the frequency and the coding pattern is instructed to the coding means 201 according to the coding pattern. In order to determine the coding pattern, the coding parameter determining means 202 uses the included coding pattern reference table 210.

Note that the video encoding device according to the second embodiment is also realized by executing the encoding program on the PC as in the first embodiment, and the condition (1) described in the first embodiment is satisfied. ) To (5). It is assumed that the operating frequency of the CPU mounted on this apparatus is either 100 MHz or 166 MHz, the frame rate specified at the start of encoding is 24 frames / sec, and the resolution of the frame image in the input image is , 320 × 240 or 160 × 120.

The operation of the video encoding apparatus according to the second embodiment configured as described above under the above settings will be described below. Input image data is input for each frame image, and the encoding unit 201 performs an encoding process on the input image data. The operation of the encoding unit 201 is the same as that of the encoding unit 101 shown in the first embodiment.

On the other hand, the processing capability determining means 211 determines the processing capability of the present apparatus by using a P which implements the present apparatus.
The operating frequency of the CPU mounted on C is detected, and this is notified to the encoding parameter determining means 202 as a determination result. The judgment result indicating either the operating frequency of 100 MHz or 166 MHz is input to the coding parameter determining means 202, and the coding parameter determining means 202
Determines an encoding parameter using the result of this determination.

Table 4 shows the encoding parameter determining means 20.
2 is a table showing an encoding pattern reference table 210 included in the encoding pattern reference table 210; FIG. 5 shows an encoding parameter determining means 2
It is a flowchart figure which shows operation | movement of 02. Hereinafter, the operation of the encoding parameter determination unit 202 will be described with reference to Table 4 and in accordance with the flow of FIG.

[0162]

[Table 4]

Similarly to the resolution reference table 110 included in the encoding parameter determination means 101 in the first embodiment, the encoding pattern reference table 2 in (Table 4) is used.
Numeral 10 is created in advance before encoding in consideration of conditions described later.

The relationship between the "input" column and the "output" column shown in (Table 4) is the same as in (Table 1), and the set frame rate and resolution, and the operating frequency which is the input judgment result The coding pattern is determined corresponding to the three. Here, regarding the encoding pattern,
“IIIIII” means that intra-frame coding (I) is performed on all frame images, and “IPIPI”
“P” means that the intra-frame coding (I) and the forward prediction coding (P) are repeated every two frames.
“IP” means that intra-frame coding (I) is performed every 5 frames.
Iteratively repeating the process of performing forward prediction coding (P) once after repeating "IPPPP"
"P" means that the process of repeating intra-frame coding (I) once every six frames and then repeating forward prediction coding (P) five times is repeated.

As in the case of setting the resolution reference table in the first embodiment, the creation of the reference table is performed in consideration of the following conditions. First, the forward predictive encoding process requires a large amount of processing compared to the intra-frame encoding process, but can be encoded at a high compression rate. Second, it encodes a frame image at high resolution when encoding. In that case, the processing amount is larger than that in the case of encoding at low resolution. Third, the higher the operating frequency of the CPU, the higher the processing capacity.
The condition is that the encoding process can be executed in a short time. In consideration of these conditions, the encoding pattern reference table 110 is created so that the encoding process can be executed at the highest possible compression ratio while achieving the specified frame rate.

When the judgment result is input from the processing ability judgment means 211, the coding parameter determination means 202 operates according to the flow shown in FIG. First, in step C01, the encoding parameter determination means 202 determines the designated frame rate (24 frames / second) and the resolution (320 × 24
0 or 160 × 120) and the processing capacity determination means 21
The coding pattern at the time of executing the coding is determined by referring to the coding pattern reference table 210 based on the CPU operating frequency (100 MHz or 166 MHz) input by 1.

Subsequently, step C02 is executed, and the coding parameter determining means 202 outputs coding parameters to the coding means 201. The coding parameter determining means 202 specifies the coding type (I or P) so as to realize the coding pattern determined in step C01, and also specifies the designated resolution. Thereafter, it is determined in step C03 whether or not encoding has been completed. If it is determined that encoding has been completed, the process ends. On the other hand, if the processing has not been completed, the process returns to step C02, whereby the output of the coding parameters to the coding means 201 is repeated.

Encoding is performed by the above-described operations of the encoding unit 201, the encoding parameter determining unit 202, and the processing capability determining unit 211 (Table 5).
9 is a table showing the result of encoding performed by the video encoding device according to the second embodiment.

[0169]

[Table 5]

[0170] In (Table 5), the result of the judgment on the specified coding condition and the processing capability and the coding pattern which is the decision parameter determined from the above are the same as those in (Table 4). In the case of, the frame rate and the amount of encoded data obtained as a result of the encoding process (encoding result) are shown. Here, the encoded data amount is 1 in the input image.
When the data amount of one frame image is 1, the data amount of the encoded data corresponding to one frame image in the encoded data is shown. That is, the smaller the encoded data amount, the higher the compression ratio.

[0171] In Table 5 also, the first embodiment is used.
As in the case of (Table 2), the frame rate of the encoding result is calculated as follows. That is, CPU
Operating frequency is 100 MHz, and the coding pattern “IIII
II ", based on the fact that processing can be performed at 24 frames / sec when the resolution is 320 × 240, the frame rate in other cases is calculated. For example, when the CPU is 166 MHz, the coding pattern is “IIIIIP”, and the resolution is 320 × 240, the frame rate is that the processing of P takes six times as long as the processing of I. Can be processed in 100/166 of the time, so that (5/24 + 6/24) × (100)
/166)=0.276 seconds can be calculated, from which it can be calculated to be 21.731 frames / second. Similarly, the data amount of encoding in one frame image shown as an encoding result is 1 unit when encoded by I.
/ 10, and 1/60 when encoded with P. For example, when encoding is performed with the pattern “IIIIIP”, the encoded data amount of six frame images is (5/10 + 1/60)
= 0.517, the encoded data amount for one frame image is 0.086.

For comparison, Table 6 shows an encoding result when encoding is performed using a conventional video encoding device.

[0173]

[Table 6]

In Table 6 also, the operating frequency of the CPU is 100 MHz, and the coding pattern “IIIIII”
Based on the fact that processing can be performed at 24 frames / sec when the resolution is 320 × 240, the frame rate as the encoding result in other cases is calculated. Also, the amount of encoded data is 1/10 when one frame image is encoded with I, and P
Is calculated based on 1/60 when encoded by.

In the video coding apparatus according to the prior art, the frame rate obtained as a result of coding or the fluctuation of the hardware capability of the coding apparatus is not taken into consideration,
The encoding type (pattern) or the resolution has been determined. Therefore, there is a problem that the frame rate obtained as a result of the encoding process according to the setting sometimes becomes an unnecessary numerical value. On the other hand, in the video encoding device according to the second embodiment, in consideration of the processing capability of the encoding device and the frame rate as an encoding result, the encoding type (pattern ), A frame rate close to the specified frame rate can be realized as shown in the comparison of Table 5 and Table 6, and encoding at a higher compression rate is performed. I understand.

In particular, the advantage of being able to cope with fluctuations in the hardware capabilities of the encoding apparatus is that by executing a video encoding program on a computer or the like,
This is useful when implementing the video encoding device.

As described above, according to the video coding apparatus of the second embodiment, the coding means 201, the coding parameter determination means 202 including the coding pattern reference table 210, and the processing capability determination means 211 Is provided, the encoding parameter determination unit 202 determines that the designated frame rate and resolution, and the processing capability determination unit 211
The encoding pattern is determined in accordance with the judgment result output by the encoding unit, and the encoding parameter is output to the encoding unit 201.
Since the encoding unit 201 performs the encoding process according to the encoding parameter, it is possible to perform the encoding with a higher compression ratio while realizing the required conditions.

In the video encoding apparatus according to the second embodiment, the encoding pattern is determined according to the designated resolution. However, by using the same lookup table, the designated encoding is performed. It is also possible to determine the resolution in accordance with the pattern (type), and it is possible to perform encoding processing at a higher resolution under the required frame rate and encoding pattern.

In the second embodiment, the processing capability is determined based on the operating frequency of the CPU. However, the processing capability of the CPU or the processor such as a DSP, such as the product number, version, and the manufacturer, is determined. May be determined on the basis of various factors indicating the above, and various applications are possible.

Embodiment 3 The video encoding method according to the third embodiment of the present invention determines an encoding parameter based on a set frame rate in accordance with the processing capability of the encoding apparatus, and determines the encoding parameter based on a required processing time. This is to determine the processing capacity.

FIG. 6 is a block diagram showing a configuration of a video encoding apparatus according to Embodiment 3 of the present invention. As shown in the figure, the video encoding apparatus according to the third embodiment includes an encoding unit 301, an encoding parameter determining unit 302, and a processing capacity determining unit 311. Processing means 303, quantization means 30
4. Variable length encoding means 305, bit stream generation means 306, inverse quantization means 307, inverse DCT processing means 30
8 and the predicted image generation means 309, and the coding parameter determination means 302
0 is included. Encoding means 301 is similar to that of the first embodiment.
Is the same as the encoding unit 101 of the video encoding device according to the above, and corresponding to the encoding parameter input from the encoding parameter determining unit 302, the input frame image is instructed at the indicated resolution, and Intra-frame coding
Encoding is performed using an indicated encoding type such as (I) or forward prediction encoding (P).

The coding parameter determining means 302 determines coding parameters using the included coding pattern determining means 310 in accordance with the result of the determination by the processing capacity determining means 311, and outputs the determined coding parameters to the coding means 301. The processing capability determining unit 311 determines the encoding processing capability of the encoding device, and outputs a result of the determination to the encoding parameter determining unit 302. In the third embodiment, the processing capacity determination unit 311
Outputs the average frame rate of the encoding process in the encoding apparatus as a determination result. The encoding parameter determination unit 302 determines the designated frame rate, the resolution, and the average frame rate as the determination result. To determine the coding type to the coding means 301 according to the coding pattern. In order to determine the coding pattern, the coding parameter determining means 302 uses the coding pattern determining means 310.

It is to be noted that, similarly to the first embodiment, the video encoding apparatus according to the third embodiment is realized by executing the encoding program on the PC, and the condition (1) described in the first embodiment is satisfied. ) To (5). Further, it is assumed that the operating frequency of the CPU mounted on this apparatus is 100 MHz. The frame rate specified at the start of encoding is 8 frames / second, and the resolution of the frame image in the input image is 320 × 240.
Is specified.

The operation of the video encoding apparatus according to the third embodiment configured as described above under the above settings will be described below. Input image data is input for each frame image, and the encoding unit 301 performs an encoding process on the input image data. The operation of the encoding unit 301 is the same as that of the encoding unit 101 shown in the first embodiment.

On the other hand, in order to determine the processing capability of the present apparatus, the processing capability determining means 311 calculates the processing capability of all the preceding frame images, including the four frame images, every four frame images. The encoding unit 301 measures the time required for processing, calculates the average frame rate of the encoding processing up to that time from the measured processing required time and the number of processed frame images, and calculates an encoding parameter. Notify the determination means 302. It is assumed that the processing capacity determination means notifies the requested frame rate of 8 frames / second as the initial value of the average frame rate.

The coding parameter determining means 302 determines a coding parameter using the notified average frame rate. The operation of the coding pattern determining means 310 included in the coding parameter determining means 302 will be described below. The coding pattern determination means 310 takes a limited number of states, and operates as a finite state machine that transitions between these possible states according to conditions. FIG. 7 is a state transition diagram showing a state transition (a) in the coding pattern determination means 310 as a finite state machine, and (b) a diagram showing a state transition condition. The coding pattern determination means 310 takes four states in total from S0 to S3, and outputs a coding pattern shown in (Table 7) in each state.

[0187]

[Table 7]

Regarding the encoding patterns shown in Table 7, “IIII” indicates that intra-frame encoding (I) is to be performed on all of the four frame images to be processed. "IIIP" indicates that the first three of the four frame images are subjected to intra-frame encoding (I), and the last one is subjected to forward prediction encoding (P). “IPIP” executes intra-frame coding (I) for the first and third frames of the four frame images, and performs forward prediction coding (P) for the second and fourth frames. To do, “IPPP” is one of the four frame images. For the first one, this means performing intra-frame coding (I), and for the remaining three, performing forward prediction coding (P).

The state transition of the encoding pattern determining means 310 is to determine the state transition for each of the four frame images. The determination is made by the processing capability determining means 311 immediately before the determination. Using the notified value of the average frame rate and the designated value of the frame rate, the processing is performed according to the conditions shown in FIG. However, in the third embodiment, the initial value of the state taken as the finite state machine is S1.

As described above, in the video encoding apparatus according to the third embodiment, when encoding is started, first, the initial value “8 frames / sec” is output from the processing capability judging unit 311 and the encoding pattern determining unit Since 310 is in the initial state S1, "IIIP" is output as an encoding pattern as shown in Table 7. Accordingly, the coding parameter determination means 302 outputs the coding parameters to the coding means 301 so as to realize the pattern, and performs intra-frame coding for three frame images and forward coding for the next one. Control is performed so that predictive coding is performed.

Thereafter, when the average frame rate obtained from the processing capability judging means 311 becomes lower than the designated frame rate, a transition from S1 to S0 is made as shown in FIG. As a result, the encoding pattern shown in Table 7 is changed to "IIII", and only intra-frame encoding is performed.

On the other hand, when the average frame rate obtained from the processing capability judging means 311 is higher than the designated frame rate, the transition from S1 to S2 shown in FIG. 7A is made, and the encoding pattern is changed to "IPIP". To increase the ratio of forward prediction coding.

By performing such control, when the average frame rate output from the processing capability judging means 311 is smaller than the designated frame rate, the processing load of the encoding apparatus is considered to be heavy. FIG.
The transition in the S3 → S0 direction shown in (a) increases the ratio of intra-frame encoding with a small processing load in the encoding process, while increasing the frame rate specified by the average frame rate output from the processing capability determination unit 311. When the rate is larger than the rate, it is considered that the processing capability of the encoding apparatus has a margin. Therefore, the transition in the S0 → S3 direction shown in FIG. The ratio of the predictive encoding is increased so that an encoding result with a higher degree of compression can be obtained.

As described above, the encoding is executed while the encoding parameters are changed in accordance with the state of the encoding process indicated as the processing capability. 25 is a table illustrating a result of encoding when encoding is performed on 28 continuous frame images.

[0195]

[Table 8]

In the table, the designated frame rate is fixedly set to “8”. The values of the encoding pattern, the required time, and the output of the processing capability determination unit 311 are shown for each of the four frame images such as the 0th to 3rd, the 4th to 7th, etc. . The coding pattern is a coding pattern used for the coding process for each of the four frame images, the required time is a time (second) required for the coding unit 301 to perform the coding process on the four frame images, and a processing capability determination. The output of the means 311 indicates the average frame rate obtained using the required time. As a result of encoding the 28 frame images, the average frame rate and the encoded data amount in the encoding process are shown.

In Table 8, when the operating frequency of the CPU is 100 MHz and the encoding pattern “III” has a resolution of 320 × 240, processing can be performed at 24 frames / sec. Are calculated, and a frame rate as an encoding result is calculated. For example, the time required to process the four frame images from the 0th to the 3rd is
1/24 × 3 + 6/24 × 1 = 0.375 seconds. In addition, since 15 I-frames and 13 P-frames are generated as encoding results, the time required for encoding one frame image is (1/24 × 15 + 6/24 × 1).
3) ÷ 28 = 0.138 seconds, and the average frame rate can be calculated to be 7.225 frames / second. Also, the encoded data amount of one frame image in the encoding result is:
It is calculated based on the fact that it becomes 1/10 when encoded with I and 1/60 when encoded with P. As a result, as an encoding result, the I frame
Since 13 frames and 13 P frames are generated, the encoded data amount of 28 frame images is (15/10 + 13/6)
0), the encoded data amount for one frame image is 0.061.

(Table 9) is a table for explaining the functions of the coding pattern determining means 310 and the coding parameter determining means 302 in the video coding apparatus according to the third embodiment. In the same table, the encoding pattern determination unit 310 and the encoding code for each of 12 frame images from the 0th to the 11th among the 28 frame images to be processed in the above (Table 8) 14 shows the state of the video encoding device according to the third embodiment, including the output of the encoding parameter determination means 302.

[0199]

[Table 9]

In the table, the required time and the output of the processing capacity judging means 311 are the same as in (Table 8). In response to the output of the processing capability determination unit 311, the encoding pattern determination unit 310 makes a state transition and outputs an encoding pattern according to the state. Then, the encoding parameter determination unit 302 outputs the encoding type for each frame image, as shown in the table, corresponding to the output encoding pattern.

As shown in the table, since the state of the coding pattern determining means 310 first takes S1 as described above,
Therefore, as shown in Table 7, the coding pattern “IIIP”
Is output. The encoding parameter determination means 302 responds by responding to the “I” for the first four frame images.
"I", "I" and "P" are output.

For the four frame images from the eighth to the eleventh, since the state of the coding pattern determining means 310 has changed from S1 to S2, the coding pattern "I
Since “PIP” is output, the output of the encoding parameter determination unit 302 is “I”, “P”, “I”, and “P”.

As shown in Table 8, each time four frame images are processed, the processing capacity is determined, and a coding pattern corresponding to the processing capacity is selected, and a coding type corresponding to the pattern is selected. Processing is performed.

For comparison, Table 10 shows that 28 frame images were obtained by using a video encoding device according to the related art.
7 shows an encoding result when encoding is performed using a predetermined encoding pattern.

[0205]

[Table 10]

As the encoding patterns, the four patterns shown in (Table 7) are used. As the encoding result, the frame rate of the encoding process, the encoded data amount for one frame image, Is shown. Note that also in Table 10, the operating frequency of the CPU is 100 MHz,
Since the encoding pattern IIII can be processed at 24 frames / sec when the resolution is 320 × 240, the frame rate in other cases is calculated. Also, regarding the amount of encoded data, for one frame image,
It is calculated based on the fact that it becomes 1/10 when encoded with I and 1/60 when encoded with P.

[0207] In a video encoding device according to the prior art, a frame rate obtained as a result of encoding or a change in hardware capability constituting the encoding device is not taken into consideration,
The encoding type (pattern) or the resolution has been determined. Therefore, it is difficult to set the frame rate obtained as a result of the encoding process to be close to a desired value. In some cases, a setting that results in an unnecessary numerical value must be selected. Was. On the other hand, in the video encoding device according to the third embodiment, in consideration of the processing capability of the encoding device and the fluctuation of the frame rate as an encoding result, the encoding type ( By determining the pattern, a frame rate close to the designated frame rate can be realized as shown in a comparison between Table 9 and Table 10, and encoding at a higher compression rate is executed. You can see that.

As described above, according to the video coding apparatus of the third embodiment, the coding means 301, the coding parameter determining means 302 including the coding pattern determining means 310, and the processing capability determining means 311 By having
The encoding parameter determining unit 302 determines an encoding pattern in accordance with the specified frame rate and resolution, and the determination result output from the processing capability determining unit 311, and outputs the encoding parameter to the encoding unit 301. However, since the encoding unit 301 performs the encoding process according to the encoding parameter, it is possible to perform the encoding with a higher compression ratio while realizing the required conditions.

[0209] In the video encoding apparatus according to the third embodiment, the encoding pattern is determined in accordance with the specified resolution. However, by performing the same processing, the specified encoding pattern is determined. It is also possible to determine the resolution corresponding to (type), and it is possible to perform encoding processing at a higher resolution under the required frame rate and encoding pattern.

The video coding apparatus according to the first and second embodiments basically determines a coding parameter at the start of coding, and thereafter performs a coding process according to the determined coding parameter. However, in the device according to the third embodiment, the average rate is calculated while performing the encoding process, and the encoding parameter is dynamically changed according to the encoding situation acquired as the processing capability of the device. Can be done. Therefore, in the video encoding device according to the third embodiment, although a small processing load is involved as compared with the first and second embodiments, a general-purpose computer that executes a plurality of arithmetic processes and the like in parallel performs video capturing. In such a case as when performing the encoding accompanying the above, even when the situation of the computer device changes, it is possible to set appropriate encoding conditions in response to the change of the situation. It is.

However, there is not much change in the situation.
For example, when the processing capability of the encoding device is not expected to change significantly during the encoding process, the video encoding device according to the third embodiment can also use the first and second embodiments.
As in the case of (2) and (2), parameters are set at the start of encoding, and thereafter, encoding is performed under the conditions.
It is also possible to reduce the processing load on the control.

Embodiment 4 The video encoding method according to the fourth embodiment of the present invention determines encoding parameters based on a set frame rate in accordance with the processing capability of the encoding device, and the amount of temporarily stored data. Is used to determine the processing capacity.

FIG. 8 is a block diagram showing a configuration of a video encoding device that executes the video encoding method according to Embodiment 4 of the present invention. As shown, the video encoding apparatus according to the fourth embodiment includes an encoding unit 401, an encoding parameter determination unit 402, a processing capability determination unit 411, a buffer unit 412, and an input frame rate control unit 41.
3 is comprised. The encoding unit 401 includes a DCT processing unit 403, a quantization unit 404, and a variable length encoding unit 4
05, a bit stream generation unit 406, an inverse quantization unit 407, an inverse DCT processing unit 408, and a predicted image generation unit 409, and an encoding parameter determination unit 402
Includes an encoding pattern determination unit 410.

The coding means 401 is the same as the coding means 101 of the video coding apparatus according to the first embodiment, and corresponds to the coding parameters input from the coding parameter determining means 402, For images,
At the indicated resolution and intra-frame encoding (I),
Alternatively, encoding is performed using a designated encoding type such as forward prediction encoding (P). In Embodiments 1 to 3,
All of the encoding units 101 to 301 input the input image data. However, in the fourth embodiment, the encoding unit 401 reads out data from a buffer unit 412 described later and performs an encoding process. is there.

[0215] The coding parameter determining means 402 determines coding parameters in accordance with the judgment result of the processing capability judging means 411, and outputs it to the coding means 401. The processing capability determination unit 411 determines the encoding processing capability of the encoding device, and outputs a result of the determination to the encoding parameter determination unit 402. In the fourth embodiment, the processing capacity determining unit 41
Reference numeral 1 denotes an output of a buffer amount temporarily stored in a buffer unit 412 described later as a determination result. Also,
The encoding parameter determining unit 402 determines an encoding pattern from the specified frame rate, resolution, and buffer amount as a result of the determination, and instructs the encoding unit 401 of the encoding type according to the encoding pattern. Is what you do. In order to determine the coding pattern, the coding parameter determining means 402 uses the coding pattern determining means 410.

The input frame rate control means 413 outputs the input image data, which is the input of the video encoding device, as a series of frame images to a buffer means 412, which will be described later, corresponding to the specified frame rate. The buffer unit 412 temporarily stores the input image data, sequentially stores the input image data as a series of frame images, and sequentially discards the frame images read by the encoding unit 401. Then, in the fourth embodiment, the processing capability determination unit 411
A method for detecting a difference between the number of frame images stored in the buffer unit 412 at the start of encoding and the number of currently stored frame images and outputting the difference to the encoding parameter determining unit 402 as a determination result It is.

Note that the video encoding device according to the fourth embodiment is also realized by executing the encoding program on the PC as in the first embodiment, and the condition (1) described in the first embodiment is satisfied. ) To (5). Further, it is assumed that the operating frequency of the CPU mounted on this apparatus is 100 MHz. The frame rate specified at the start of encoding is 8 frames / second, and the resolution of the frame image in the input image is 320 × 240.
Shall be specified.

The operation of the video encoding apparatus according to the fourth embodiment configured as described above under the above settings will be described below. When a video to be processed by the video encoding device according to the fourth embodiment is input as input image data, the input image data is first input to input frame rate control means 413. The input frame rate control unit 413 converts the input image data into a series of frame images at a designated frame rate.
12 sequentially. In the fourth embodiment,
The input frame rate control means 413 performs processing with the above-mentioned 8 frame / sec as the specified frame rate.

The buffer unit 412 sequentially stores the frame images input from the input frame rate control unit 413, and sequentially discards the frame images read by the encoding unit 401. That is, data is temporarily stored by a FIFO (first in first out) method.
At the start of encoding, it is assumed that the encoding unit 401 starts operating with a certain time delay from the input frame rate control unit 413. That is, at the start of encoding, the buffer unit 412 stores a certain number of frame images. This prevents the processing from proceeding smoothly due to buffer underflow, that is, depletion of temporarily stored data. At this stage, the processing capacity determination unit 411 detects the number of frame images stored in the buffer unit 412 in the initial state,
Retained for comparison processing.

The encoding means 401 comprises a buffer means 412
The temporarily stored frame image is read out, and encoding processing is performed. The operation of the encoding unit 401 at the time of the encoding process is the same as that of the encoding unit 101 in the first embodiment. On the other hand, each time the encoding unit 401 processes four frame images,
The number of frame images stored in the buffer unit 412 is detected, the difference between the number of frame images in the buffer unit 412 in the initial state, which is detected and held first, is obtained, and this difference value is encoded. Parameterization means 4
02 is notified. At this time, when the number of stored frame images is larger than the number of frame images in the initial state, the difference is notified as a positive value, and when the number is smaller, the difference is notified as a negative value.

In the coding parameter determining means 402, the above difference is input to the coding pattern determining means 410. The coding pattern determining unit 410 operates as a finite state machine, like the coding pattern determining unit 310 in the third embodiment. FIG. 9A is a state transition diagram of the coding pattern determining means 410 operating as a finite state machine, and FIG. 9B is a diagram showing a state transition condition. The coding pattern determination means takes a total of four states from S0 to S3, and in each state, (Table 1
The coding pattern shown in 1) is output.

[0222]

[Table 11]

The “IIII” and “II” shown in (Table 11)
The patterns of “IP”, “IPIP”, and “IPPP” are the same as those shown in (Table 7) of the third embodiment. The state transition in the encoding pattern determination unit 410 is determined each time the encoding unit 401 processes four frame images, and the determination is notified from the processing capability determination unit 411. The determination is performed based on the difference value immediately before the determination in accordance with the condition shown in FIG. In the fourth embodiment, it is assumed that the initial value of the state as the finite state machine is S1.

Each time the encoding means 401 processes four frame images, the encoding parameter determination means 402
The encoding unit 401 obtains the encoding pattern output from the encoding pattern determining unit 410, and encodes the encoding unit 401 for each frame image so that the acquired encoding pattern can be realized.
To indicate the encoding type. Further, the designated resolution is directly instructed to the encoding means 401.

As described above, in the video coding apparatus according to the fourth embodiment, when coding is started, first, since the coding pattern determining means 410 is in the initial state S1,
As shown in Table 12, "IIIP"
Is output. Therefore, the encoding parameter determining means 402
Outputs encoding parameters to the encoding means 401 so that the pattern can be realized, and performs intra-frame encoding on three frame images and forward prediction encoding on the next one. Control is exercised.

Thereafter, when the difference obtained from the processing capacity judging means 411 becomes a negative value, a transition from S1 to S0 is made as shown in FIG.
The coding pattern shown in FIG. 2 is changed to “IIII”, and only intra-frame coding is performed. On the other hand, when the difference value obtained from the processing capacity determination means 411 becomes a positive value, the transition from S1 to S2 shown in FIG. 9A is made, the encoding pattern is changed to "IPIP", and the forward direction is changed. The ratio of predictive coding will increase.

By performing such control, when the difference value output from the processing capacity determining means 411 is positive, that is, when the number of frame images stored in the buffer means 412 is larger than the initial number of stored images, Since the processing load of the encoding apparatus is considered to be heavy, the ratio of intra-frame encoding with a small processing load in the encoding processing is increased by the transition from S3 to S0 shown in FIG. Aim at. On the other hand, when the difference value output from the processing capability determination unit 411 is, that is, when the number of frame images stored in the buffer unit 412 is larger than the initial storage number, the processing capability of the encoding device has a margin. 9A, the ratio of forward predictive coding with a large processing load in the encoding process is increased by the transition in the S0 → S3 direction shown in FIG. It is intended to obtain a result.

As described above, the encoding is executed while the encoding parameter is changed in accordance with the state of the encoding process indicated as the number of stored images. 25 is a table illustrating a result of encoding when encoding is performed on 28 continuous frame images.

[0229]

[Table 12]

In the table, the designated frame rate is fixedly set to “8”. The encoding pattern, the required time, the input number, the output number, and the output of the processing capability determination unit 411 are 0th to 3rd,
The values are shown for each of the processing of four frame images such as the seventh to seventh. The encoding pattern is the encoding pattern used for the encoding process for each of the four frame images, the required time is the time (seconds) required for the encoding unit 401 to encode the four frame images, and the input number is The number of frame images input to the buffer unit 412 and the number of output frames indicate the number of frame images output from the buffer unit 412. And 28
As a result of encoding one frame image, an average frame rate and an encoded data amount in the encoding process are shown.

In Table 12, four frames are used based on the fact that the processing frequency of the CPU is 100 MHz and the coding pattern “III” can be processed at 24 frames / sec when the resolution is 320 × 240. The time required to process the image and the frame rate as an encoding result have been calculated.

For example, the time required to process the four frame images from the 0th to the 3rd is 1/24 × 3
+ 6/24 × 1 = 0.375 seconds. As the encoding result, fifteen I frames and thirteen P frames are generated, so the time required for encoding one frame image is (1/24 × 15 + 6/24 × 13) ÷ 28 =
0.138 seconds, the average frame rate is 7.225
It can be calculated as frames / second. Further, the encoded data amount of one frame image in the encoding result is 1/10 when encoded by I, and 1 when encoded by P.
/ 60, respectively. As a result, as an encoding result, 15 I frames and P
Since 13 frames are generated, the encoded data amount of 28 frame images is (15/10 + 13/60) =
Since it is 1.717, the encoded data amount for one frame image is 0.061.

Table 13 shows the coding pattern determining means 410 in the video coding apparatus according to the fourth embodiment.
4 is a table for explaining the function of a coding parameter determination unit 402. In the table, among the 28 frame images to be processed in the above (Table 12), the encoding pattern determination unit 410 and the encoding code for each of the 12 frame images from the 0th to the 11th are shown. 13 shows the state of the video encoding device according to the third embodiment, including the output of the encoding parameter determination means 402.

[0234]

[Table 13]

In the table, the required time, the number of input sheets, the number of output sheets, and the output of the processing capability judging means 411 are as shown in (Table 1).
Same as 2). In response to the output of the processing capability determining unit 411, the coding pattern determining unit 410 makes a state transition as described above and outputs a coding pattern. Then, the encoding parameter determining means 302 outputs the encoding type for each frame image as shown in the table, corresponding to the outputted encoding pattern.

As shown in the table, since the state of the coding pattern determining means 410 first takes S1 as described above,
Therefore, as shown in Table 11, the coding pattern “III
P "is output. In response to this, the encoding parameter determination unit 402 outputs “I”, “I”, “I”, and “P” for the first four frame images.

With respect to the four frame images from the eighth to the eleventh, the state of the coding pattern determining means 410 has changed from S1 to S2.
Since “PIP” is output, the output of the encoding parameter determination unit 402 is “I”, “P”, “I”, and “P”.

As shown in Table 12, each time four frame images are processed, the processing capability is determined and the corresponding coding pattern is selected. Processing is performed.

In the video encoding device according to the prior art, as shown in (Table 10) in the third embodiment,
The encoding type (pattern) or the resolution is determined without considering the frame rate obtained as the encoding result or the fluctuation of the hardware capability of the encoding apparatus. Therefore, it is difficult to set the frame rate obtained as a result of the encoding process close to a desired value, and sometimes it is necessary to select a setting that results in an unnecessary value. Was. On the other hand, in the video encoding device according to the fourth embodiment, similarly to the third embodiment, the video encoding device is designated in consideration of the processing capability of the encoding device and the fluctuation of the frame rate as the encoding result. By determining the encoding type (pattern) according to the resolution, a frame rate close to the specified frame rate can be realized as shown in a comparison between Table 13 and Table 10, and at a higher compression rate. It can be seen that the encoding has been performed. Embodiment 3
, It is necessary to measure the time required for the processing. However, in the fourth embodiment, even if the processing time cannot be measured or difficult as in the third embodiment, The processing capacity of the apparatus can be determined using the temporarily accumulated data amount as an index.

As described above, according to the video coding apparatus of the fourth embodiment, the coding parameter determining means 4 including the coding means 401 and the coding pattern determining means 410
02, processing capacity determination means 411, buffer means 412,
And the input frame rate control means 413, the input frame rate control means 413 inputs the input image data to the buffer means 412 at a predetermined rate, and the input buffer means 412 temporarily stores the input image data. Then, the processing capability determination unit 411 outputs a determination result indicating the processing capability of the encoding apparatus based on the amount of data stored in the buffer unit 412, and the encoding parameter determination unit 402 , A resolution, and a determination result output from the processing capability determination unit 411, an encoding pattern is determined, and encoding parameters are output to the encoding unit 401. Since the encoding process is performed, it is necessary to perform encoding that achieves a higher compression rate while achieving the required conditions. It can become.

In the video coding apparatus according to the fourth embodiment, the coding pattern is determined in accordance with the specified resolution. However, by performing the same processing, the specified coding pattern is determined. It is also possible to determine the resolution corresponding to (type), and it is possible to perform encoding processing at a higher resolution under the required frame rate and encoding pattern.

Also, in the device according to the fourth embodiment, as in the third embodiment, the amount of data temporarily stored is detected while performing the encoding process, and the amount of encoded data acquired as the processing capability of the device is detected. The coding parameters can be dynamically changed according to the situation. Therefore, in the video encoding device according to the fourth embodiment, although a small processing load is involved as compared with the first and second embodiments, a general-purpose computer that executes a plurality of arithmetic processes and the like in parallel performs video capturing. In such a case as when performing the encoding accompanying the above, even when the situation of the computer device changes, it is possible to set appropriate encoding conditions in response to the change of the situation. It is.

However, as in the case of the third embodiment, in the case where it is expected that the processing capacity of the coding apparatus does not fluctuate significantly in the course of the coding processing, such as when the situation does not change much, the fourth embodiment is used. As in the first and second embodiments, the video encoding apparatus according to the first embodiment sets parameters at the start of encoding, and thereafter performs encoding under the same conditions, thereby reducing the processing load on control. Is also possible.

In Embodiments 1-4, intra-frame encoding and forward prediction encoding are performed as encoding types, but the encoding types are not limited to these. For example, backward predictive coding, bidirectional predictive coding, etc., can also be adopted as different coding types, respectively.Moreover, even when the search range of a motion vector is changed in inter-frame predictive coding,
It can be used as a different encoding type.

In the video encoding method described in the first to fourth embodiments, a personal computer, a workstation, or the like uses a recording medium storing a video encoding program capable of executing the method. Can be realized by executing

[0246] Embodiment 5 The speech encoding device according to the fifth embodiment of the present invention can reduce the processing load on the speech encoding device by performing conversion processing on the sampled speech data. FIG.
Is a block diagram illustrating a configuration of a speech coding apparatus according to a fifth embodiment of the present invention, and FIG. 11 is a diagram illustrating a hardware configuration of the coding apparatus according to the fifth embodiment. As shown in FIG. 10, the audio encoding device includes an audio input unit 501, a register 502, an input audio sampling unit 503, an audio data conversion unit 504, a band division unit 505, an encoded bit allocation unit 506, and a quantization unit 507. , An encoding unit 508, and an encoded data recording unit 509.

[0247] The audio input unit 501 is for inputting audio to be encoded. The sound may be input from a microphone as shown in FIG. 11, or may be a line input. The register 502 is realized by the main memory or the external storage device of FIG. 11, and stores a constant used for the encoding process. Input audio sampling unit 50
3 is realized by the sound board (input) and the control program in FIG. 11, and performs sampling processing on the sound input by the sound input unit 501.

The audio data conversion unit 504 performs a conversion process on the data sampled by the input audio sampling unit 503 using the constant value stored in the register 502. The band dividing unit 505 includes the audio data converting unit 5
04 performs band division on the converted data. The coded bit allocation unit 506 allocates coded bits to the band divided by the band division unit 505. Quantizer 507
Performs a quantization process according to the number of coded bits allocated by the coded bit allocation unit 506. Encoding unit 508
Outputs the quantization value output from the quantization unit 507 as encoded audio data. 504 to 508 are all the same
This is realized by one CPU, main memory, and program. The encoded data recording unit 509 is realized by the external storage device and the control program of FIG.
The encoded data output from 8 is recorded as the result of the audio encoding process of the audio encoding device.

In the fifth embodiment, the set frequency fs
Is 32 kHz specified by MPEG Audio.
It is assumed that 48 kHz is adopted among three sampling frequencies of z, 44.1 kHz and 48 kHz. or,
It is assumed that the conversion constant n is stored in the register 502 as a value “2” predetermined according to the CPU performance. Regarding the determination of the value of the conversion constant n, a method of setting based on the encoding processing performance, assuming a fixed CPU to be used in the apparatus, or a value determined in advance for each CPU by simulation, etc. CP by
U selection method, CP prior to encoding process
A method of performing an operation for measuring the encoding processing performance of U and setting based on the result can be used.

FIG. 12 is a flowchart showing the operation of speech encoding by the encoding apparatus according to the fifth embodiment, and FIG. 13 illustrates sampling by the encoding apparatus according to the fifth embodiment and subsequent conversion of audio data. FIG. Hereinafter, the operation at the time of encoding by the speech encoding apparatus according to the fifth embodiment will be described according to the flowchart of FIG. 12 and with reference to FIGS.

In step 1 of the flow in FIG. 12, the audio signal input from the audio input unit 501 is sampled in the input audio sampling unit 503 using the set frequency fs as the sampling frequency. As in the case of the conventional example, this sampling is performed as shown in FIG. 13A with time ts being a reciprocal relationship to the sampling frequency fs, and m pieces of sampled audio data are output.

In step 2, the audio data converter 5
04 obtains the conversion constant n stored in the register 502 and skips (n-1) pieces of m sampled audio data output from the input audio sampling unit 503 to obtain a total of m
/ N sampling audio data is extracted. In this case, since (n-1) is 1, as shown in FIG.
Sampling voice data marked with a circle every other is extracted. Then, the audio data conversion unit 504 creates a total of m pieces of converted audio data in which the extracted data are continuous n pieces each. The converted voice data as shown in FIG. 13C is output, and the converted voice data is m voice data having a frequency fs.

Here, the audio data conversion in step 2 will be considered in comparison with the case where the audio data conversion is performed by simply extracting (n-1) skips and then used for the subsequent band division coding. In this case, as shown in data C of FIG. 13D, m / n pieces of audio data having the sampling frequency fs / n are created, and the encoded audio data to be output is converted to the sampling frequency fs / n. Only the corresponding encoded data can not be reproduced as the sampling frequency fs. According to the assumption here, after sampling at 48 kHz according to the MPEG Audio regulations, n = 2
As a result, if it is assumed that band division encoding has been performed on m / 2 pieces of audio data, which are extracted one by one,
Only the encoded audio data corresponding to the sampling frequency fs / 2 = 24 kHz is output, and the MPEG Audio
o Three specified sampling frequencies, 32 kHz, 4
It can be seen that reproduction cannot be performed at any of 4.1 kHz and 48 kHz.

For this reason, in the encoding processing according to the fifth embodiment, the conversion processing in step 2 simply uses (n−
1) Not only m / n pieces of skipped audio data (data C in FIG. 13 (d)) but also converted voice data in which the same data continues n pieces at a time. FIG.
Although the actual sampling frequency of the converted audio data shown in the data B of FIG. 13C is equivalent to fs / n, FIG.
M audio data that can be handled as the sampling frequency fs in the same manner as the data A.

In step 3 subsequent to step 2 which is the above-described conversion process, band conversion section 505 divides the converted audio data into M frequency bands. In MPEG Audio band division, division into 32 bands is performed. This step is performed in the same manner as in the first conventional example.

In step 4, coded bit allocation section 5
06 stores the set frequency fs and the conversion constant n in the register 5
02, and the limit frequency fs / 2n, which is the limit of reproducibility, is calculated based on the sampling theorem generally known. Then, among the M divided bands, coded bits are allocated to a band having a frequency lower than the reproducible limit frequency, and coded bits are not allocated to a band having a frequency higher than the non-reproducible limit frequency. In this case, the number of coded bits is determined. The number of coded bits allocated is transmitted from the coded bit allocation unit 506 to the quantization unit 507.

In step 5, the quantization section 507
According to the number of coded bits allocated, the audio data is quantized for each band to output a quantized value. In step 6, the encoding unit 508 outputs coded audio data based on the quantized value. The output coded audio data is
This is recorded in the encoded data recording unit 509. As shown in the flow of FIG. 12, the above process is repeated while the input of the audio to be encoded continues, and the encoding is completed immediately after the input of the audio is completed.

Regarding the effect of the audio coding apparatus according to the fifth embodiment, the reduction of the processing amount by using the converted audio data obtained in step 2 in the band division in step 3 is described in the band division method in MPEG Audio. Consider.

In the MPEG Audio band division assumed here, the following calculation is performed for division into 32 bands.

[0260]

(Equation 1)

Xi: input audio data Si: audio data after band division Further, the coefficient Ci is based on the MPEG audio standard.
It is obtained from a coefficient table in which sample numbers and coefficients are compared.

Here, equations (1) and (2) are usually
The operation is performed on a plurality of pieces of audio data. When the operation is performed on the converted audio data in which the extracted sampled audio data converted in step 2 is continuous by n pieces,
Since the n pieces of sampled audio data are the same, the part that can continuously handle them is m /
The calculation may be performed on n pieces of audio data, and the calculation amount of Expression (1) is reduced to 1 / n, and the calculation amount of Expression (2) can be reduced to 1 / n.

Here, the case where n = 4 will be described. In this case, in the formula (1), since the voice data of X0 to X3 is a sequence of four X0s, X0 = X1
= X2 = X3, and for Ci, C0 = C1
The operation can be performed using one value such that = C2 = C3. Eventually, the four values of Z0, Z1, Z2, and Z3 are obtained by one operation, and in equation (1), all Zi are obtained with a calculation amount of 1/4. In the calculation of the equation (1), in order to represent by one value of C0 = C1 = C2 =... = Cn, a method using any value of C0 to Cn or an average value of C0 to Cn is used. A method or the like is used.

Next, also in the equation (2) using Zi, the value obtained by jumping from 0 to 64 is simply added eight times, and n is 2
In the case of exponentiation, Yi has the same value every n pieces, and the calculation is reduced to 1 / n. However, when n is not a power of 2, for example, when n = 3, Z0 added to obtain Y0 is not equal to Z65 added to obtain Y1, so that Y0 = Y1 is not obtained. Eventually, the amount of calculation of equation (2) will not be reduced.

However, even when this n is not a power of 2, it is possible to reduce the amount of calculation by setting. For example, when n = 3, X0 = X1 = X2, X3 =
In the division into 32 bands so that X4 = X5,. Perform data conversion. In other words, the last two are assumed to be the same, and the other two are converted into an audio data sequence in which consecutive triples of the same value are arranged, and the converted audio data is used to perform the band division in step 3 By performing the operations of Equations (1) and (2), a reduction close to 1/3 can be achieved.
Regarding Expression (3), the amount of calculation does not change even after the audio data conversion.

Also, in allocating the coded bits in step 4, the coded bits are allocated only to the band equal to or lower than the limited frequency fs / 2n because the voice data originally sampled at the sampling frequency fs is converted into the voice data converted in step 2. In this case, the (n-1) skipping extraction is equivalent to sampling at the sampling frequency fs / n. According to the known sampling theorem, fs
Since it can be seen that a frequency band equal to or higher than / 2n cannot be reproduced, only the band equal to or lower than fs / 2n that can be reproduced is to be encoded. Since no coded bit is assigned to a band equal to or higher than the limit frequency, and quantization for that band is unnecessary, the quantization process in step 5 is performed by １ ／
The burden is reduced to n.

As described above, according to the speech encoding apparatus of the fifth embodiment, register 502 and speech data conversion section 5
04, the input audio sampling unit 503
Converts the m sampled audio data sampled at the set frequency into an audio data conversion unit 504.
Is based on the conversion constant n stored in the register 502,
Sampling audio data is skipped by n-1 total m / n
By extracting m pieces of converted audio data, each of which is extracted and the number of the extracted sampled audio data is n, the amount of calculation can be greatly reduced in the subsequent processing by the band division unit 505, and Unlike the case where audio data is simply reduced by extraction, encoded data that can be reproduced at the original set frequency can be obtained. Also, the coded bit allocation unit 506 does not allocate coded bits to the band that cannot be reproduced by the sampling theorem for the band-divided band, and the quantization by the quantization unit 507 is unnecessary for the band. From
The burden of the quantization process is reduced to 1 / 2n. Therefore, C
In the conventional method, even when the real-time encoding process associated with audio input is difficult or impossible due to insufficient PU performance or the like, the audio encoding process is performed in real time by reducing the load by setting constants. It becomes possible.

In the fifth embodiment, the converted audio data is created in a format in which the sampled audio data is arranged. Similar effects can be obtained by inserting n-1 pieces of audio data.

Embodiment 6 FIG. The speech coding apparatus according to the sixth embodiment of the present invention can reduce the processing load on the speech coding apparatus by performing conversion processing on the sampled speech data in the same manner as in the fifth embodiment. However, the difference from the fifth embodiment is that the data amount is reduced not in the audio data conversion processing but in the sampling processing.

FIG. 14 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 6 of the present invention. As shown in the figure, the speech encoding device includes a speech input unit 601,
Register 602, input audio sampling unit 603, audio data conversion unit 604, band division unit 605, encoded bit allocation unit 606, quantization unit 607, encoding unit 608,
And an encoded data recording unit 609.
Further, the hardware configuration of the encoding device according to the sixth embodiment is the same as that of the fifth embodiment shown in FIG.

The input voice sampling section 603 differs from the input voice sampling section 503 of the first embodiment in that the set frequency is not used as the sampling frequency as it is, but a conversion constant is obtained from the register 602, and the set frequency and this conversion A sampling process is performed using a sampling frequency determined using a constant. Also, the audio data conversion unit 604 is the audio data conversion unit 104 according to the first embodiment.
Unlike sampling audio data is not extracted,
The converted voice data is created only by inserting the voice data. Voice input section 601, band division section 605, coded bit allocation section 606, quantization section 607, coding section 60
8 and the encoded data recording unit 609 are the same as 501 and 505 to 509 in the fifth embodiment. FIG. 15 is a flowchart showing an operation of speech encoding performed by the encoding device according to the sixth embodiment. Also, in the sixth embodiment, FIG. 13 is used for the description of the sampling and audio data conversion processing. Hereinafter, the operation at the time of encoding by the speech encoding apparatus according to the sixth embodiment will be described with reference to the flowchart of FIG. 15 and FIG. As in the case of the fifth embodiment, the set frequency fs is
A conversion constant n of 48 kHz specified by MPEG Audio is “2”.

In step 1 of the flow of FIG. 15, the input voice sampling unit 603 obtains the set frequency fs and the conversion constant n from the register 602, determines the effective sampling frequency fs / n from these, and 60
The audio signal input from 1 is sampled by the input audio sampling unit 603 at the effective sampling frequency fs / n. As a result of this sampling, FIG.
As shown in (d) data C, m / n pieces of sampled audio data are output.

In step 2, the audio data conversion unit 6
04 obtains a conversion constant n from the register 602, and based on the m / n sampled audio data output from the input audio sampling unit 603, converts m pieces of converted audio data in which each of the sampled audio data continues by n pieces. create.
Converted voice data such as data B in FIG. 13C is output, and the converted voice data is m voice data having a frequency fs.

As described in the fifth embodiment, the audio data of the data C shown in FIG.
Although the data cannot be reproduced in s, encoded data that can be reproduced in the sampling period fs can be obtained by converting the data into audio data such as the data B in FIG.

Since the converted voice data obtained in step 2 is equivalent to the converted voice data obtained in step 2 in the case of the fifth embodiment, subsequent steps 3 to 6 are performed in the fifth embodiment. Are performed in the same manner as in steps 3 to 6 in. Steps 1 to 6 are repeated as long as the input of the audio continues, and the encoding is completed immediately after the input of the audio is completed.

Also in the speech coding apparatus according to the sixth embodiment, the same operation as that described in the apparatus according to the fifth embodiment can be reduced in the band division stage and the quantization stage. Real-time encoding processing can be performed in accordance with CPU input and the like at a level corresponding to CPU performance and the like.

As described above, according to the speech coding apparatus of the sixth embodiment, since register 602, input speech sampling section 603, and speech data conversion section 604 are provided, input speech sampling section 603 has the set frequency. The sampling processing is performed by determining the effective sampling frequency fs / n using the fs and the conversion constant n stored in the register 602, and the audio data conversion unit 604 performs the audio processing on the obtained m / n sampled audio data. By performing the insertion of data to obtain converted audio data composed of m audio data, it is possible to reduce the processing load on the device as in the fifth embodiment. In addition, in the encoding device according to the sixth embodiment, by performing sampling at the sampling frequency fs / n, a buffer memory for temporarily storing audio data at the time of sampling input is 1 in the case of the device according to the fifth embodiment. / N, and has the advantage of being operable even when using a sound board whose sampling frequency does not have an upper limit of up to fs. It is possible to perform a real-time encoding process in response to a voice input.

In the sixth embodiment, as in the fifth embodiment, the converted audio data is created in a format in which the sampled audio data is arranged. However, n-1 suitable audio data are inserted. Thus, a similar effect can be obtained.

Embodiment 7 FIG. The speech coding apparatus according to the seventh embodiment of the present invention is intended to perform coding according to the situation by changing the conversion constant according to the amount of input data.

FIG. 16 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 7 of the present invention. As shown in the drawing, the speech encoding device includes a speech input unit 701,
Register 702, input audio sampling unit 703, audio data conversion unit 704, band division unit 705, coded bit allocation unit 706, quantization unit 707, encoding unit 708,
It comprises an encoded data recording unit 709, an input buffer 7010, and an input buffer monitoring unit 7011.
This configuration is a configuration in which an input buffer 7010 and an input buffer monitoring unit 7011 are added to the speech coding apparatus according to the fifth embodiment. Further, the hardware configuration of the encoding apparatus according to the seventh embodiment is the same as that of the fifth embodiment shown in FIG.
It is similar to that of

The input buffer 7010 is mainly realized by a memory such as a main memory, and temporarily stores data.
The input buffer monitoring unit 7011 is realized by a CPU, a main memory, and a program.
The data amount held for temporary storage at 0 is checked, this data amount is compared with a preset value, and the value of the conversion constant n of the register 702 is changed according to the result. The register 702 is the same as the register 502 of the fifth embodiment except that the value of the conversion constant to be stored is changed by the input buffer monitoring unit 7011. The input audio sampling unit 703 is the same as the input audio sampling unit 503 of the fifth embodiment except that the input audio sampling unit 703 outputs the sampled audio data to the input buffer 7010. The audio data conversion unit 704 is the same as the audio data conversion unit 504 of the fifth embodiment, except that the audio data conversion unit 704 extracts sampling audio data from the input buffer 7010 and sets it as a processing target. Also, a voice input unit 701, a band division unit 705, a coded bit allocation unit 706, a quantization unit 707, a coding unit 70
8 and the encoded data recording unit 709 are the same as 501 and 505 to 509 in the fifth embodiment.

FIG. 17 is a flowchart showing the operation of speech coding by the coding apparatus according to the seventh embodiment. The operation of the speech encoding apparatus according to the seventh embodiment during encoding will be described with reference to FIG. 17 and FIG. As in the case of the fifth embodiment, the set frequency f
s is set to 48 kHz specified by MPEG Audio. It is assumed that the conversion constant n has a value “1” predetermined according to the CPU performance stored in the register 702 as an initial value.

In step 1 of the flow of FIG. 17, the audio signal input from the audio input section 701 is sampled in the input audio sampling section 703 in the same manner as in the fifth embodiment. In step 2, the sampled audio data is input. The data is written to the buffer 7010 and temporarily stored. In step 3, the audio data conversion unit 704 reads the temporarily stored sampled audio data from the input buffer 7010. Then, after the conversion of the audio data in step 5 executed after step 4 to be described later, up to the output of the encoded data in step 9, the processing is executed in the same manner as steps 2 to 6 in the flow of FIG. Therefore, description of the operations in steps 5 to 9 will be omitted.

After Step 3 is executed, in Step 4, the input buffer monitoring unit 7011
The data amount stored in the register 10 is checked, the data amount is compared with a preset value, and the value of the conversion constant n stored in the register 702 is changed based on the comparison result. Various methods can be employed to monitor the input buffer 7010 and control the value of the conversion constant n. Here, it is assumed that the following is performed.

In the initial setting, if the encoding process cannot be performed with the voice input due to an increase in the load on the CPU or the like, the writing to the input buffer 7010 is performed at the same pace. Since the reading speed for processing is reduced, the data amount is increased.

If the data amount of the input buffer 7010 exceeds a preset buffer full level BF, the input buffer monitoring unit 7011 determines that the real-time encoding processing with the current setting is impossible. The value of the conversion constant n stored in the register 702 is increased by 1 and changed to n = 2. In steps 5 to 9 of the flow chart thereafter, in step 5, data is skipped by skipping one piece.
By converting the data into a format in which the same data continues, and performing band division in step 6, a part of the processing in the band division in step 6 is reduced to half. In step 7,
When assigning coded bits to each band, the frequency fs
By assigning coded bits only to a band equal to or less than / 4, the quantization process in step 8 is reduced to 1/4. In this way, the input buffer monitoring unit 7011 reduces the load on the CPU by changing the value of the conversion constant n.

If the data amount of the input buffer 7010 still exceeds the buffer full level BF in step 4 in the repetition of the flow of FIG. 17, the input buffer monitoring unit 7011 changes the conversion constant n of the register 2 and further Increase by 1 to n = 3. Thus, in step 5, by skipping two data and converting the data into a format in which three identical data continue, a part of the processing in the band division in step 6 is reduced to ３, and When the coded bits are allocated to each band in (4), the coded bits are allocated only to the band having a frequency equal to or lower than fs / 6, thereby reducing the quantization process in step 8 to 1/6. Thereafter, in step 4, the input buffer 7010
Until the data amount of the buffer becomes below the buffer full level BF.
The input buffer monitoring unit 7011 increases the value of n in the register 2.

Conversely, in step 4, if the amount of data held in the input buffer 7010 is lower than the preset buffer empty level BE, the input buffer monitoring unit 7011 determines that there is enough coding processing capacity. Since the smaller the value of the conversion constant n is, the higher the quality of the encoded data can be obtained without the thinning of the audio data and the cut of the high frequency component, the input buffer monitoring unit 7011
Reduces the value of the conversion constant n by 1 and thereafter repeats the flow of FIG. 17 until the data amount of the input buffer 7010 becomes equal to or more than the buffer empty level BE in the same manner as described above, and the storage of the register 702 is performed in step 4. The value of the conversion constant n is decreased by one.

In the above method, two values of the buffer full level BF and the buffer empty level BE are used to control the value of the conversion constant n. However, only the buffer full level BF may be used. In this case, the data amount of the input buffer is set to a preset buffer full level BF.
, The control is performed so that the increase of the conversion constant n is stopped when the voice input and the encoding process are balanced, that is, when the data amount reaches BF.

As described above, according to the speech coding apparatus of the seventh embodiment, the speech coding apparatus of the fifth embodiment has:
Input buffer 7010 and input buffer monitoring unit 7011
In this configuration, the sampled audio data is temporarily stored in the input buffer 7010 and then read out, and the subsequent processing is performed. The input buffer monitoring unit 7011 stores the input audio data in the input buffer 7010. By examining the amount of data, this is used as an index of the encoding processing capability of the CPU at that time, and is used as an index in the register 7.
By dynamically controlling the value of the conversion constant n stored in 02 according to the situation, it becomes possible for the CPU to perform the highest quality speech encoding that can be encoded at that time. .

[Embodiment 8] The speech coding apparatus according to the eighth embodiment of the present invention is intended to perform coding according to the situation by changing the conversion constant according to the amount of output data. FIG. 18 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 8 of the present invention. As shown in the figure, the audio encoding apparatus includes an audio input unit 801, a register 802, an input audio sampling unit 803, an audio data conversion unit 804, a band division unit 805, a coded bit allocation unit 806, and a quantization unit 8
07, an encoding unit 808, an encoded data recording unit 809, and an encoded data monitoring unit 8012.
This configuration is a configuration in which an encoded data monitoring unit 8012 is added to the speech encoding device according to the fifth embodiment. Also,
The hardware configuration of the encoding apparatus according to the eighth embodiment is also shown in FIG.
This is the same as that of the fifth embodiment shown in FIG.

The coded data monitoring unit 8012 includes a CPU,
The amount of encoded data per unit time, which is realized by the main memory and the program and output from the encoding unit 808, is checked, and this data amount is compared with a preset value.
According to the result, the value of the conversion constant n of the register 802 is changed. The register 802 is the same as the register 502 of the fifth embodiment except that the value of the conversion constant to be stored is changed by the encoded data monitoring unit 8012. Audio input unit 801, input audio sampling unit 803, audio data conversion unit 804, band division unit 805, encoded bit allocation unit 806, quantization unit 807, encoding unit 808,
The coded data recording unit 809 is the same as 501 and 503 to 509 in the fifth embodiment.

FIG. 19 is a flowchart showing the operation of speech encoding by the encoding apparatus according to the eighth embodiment. Hereinafter, the operation of the speech encoding apparatus according to the eighth embodiment during encoding will be described with reference to FIG. 19 and with reference to FIG. As in the case of the fifth embodiment, the sampling frequency fs is set to 48 k, which is defined by MPEG Audio.
Hz. It is assumed that a value “1” predetermined according to the CPU performance is stored in the register 802 as the conversion constant n as an initial value. Steps 1 to 6 of the flow in FIG. 19 are executed in the same manner as steps 1 to 6 in the fifth embodiment. In step 7, the coded data monitoring unit 8012 checks the coded data amount per unit time output from the coding unit 808, compares this data amount with a preset value, and, based on the result, , The value of the conversion constant n of the register 802 is changed. Various methods can be employed to monitor the amount of encoded data and control the value of the conversion constant n. Here, the method is performed according to the following method.

In the initial setting due to an increase in the load on the CPU or the like, if the encoding process cannot be performed in time, the encoding process slows down and the amount of encoded data output decreases. In step 7, if the encoded data amount does not reach the preset minimum encoding level CL,
The coded data monitoring unit 8012 increases the value of the conversion constant n of the register 802 as in the input buffer monitoring unit 7011 described in Embodiment 7, thereby reducing the load on the CPU. The process of FIG. 19 is repeated, and in step 7, if the coding processing amount per unit time does not fall below the highest coding level CH, the value of the conversion constant n of the register 802 is reduced so that high-quality coding can be performed. This is the same as in the seventh embodiment. Embodiment 7
Of the coded data monitoring unit 801 according to the eighth embodiment, similarly to the control by the input buffer monitoring unit 7011 in FIG.
2 also keeps changing the value of the conversion constant n until it is determined that the encoded data amount is appropriate. Also, as in the seventh embodiment, control can be performed by using only the lowest coding level CL without using the two values of the lowest coding level CL and the highest coding level CH. .

Thus, according to the speech coding apparatus of the eighth embodiment, the speech coding apparatus of the fifth embodiment has
With the configuration in which the coded data monitoring unit 8012 is added, the coded data monitoring unit 8012 checks the amount of coded data output per unit time, and determines the amount of coded data output by the CPU at that time. As an indicator,
By dynamically controlling the value of the conversion constant n stored in the register 802 in accordance with the situation, it is possible to achieve the highest quality speech encoding that can be encoded by the CPU at that time. Becomes

It should be noted that in Embodiments 7 and 8,
According to the fifth embodiment, the input audio sampling unit performs sampling at a sampling frequency fs by m
Although the audio data conversion unit performs thinning out of (n-1) skips, the input audio sampling unit determines the sampling frequency as in the case of the apparatus of the sixth embodiment. May be sampled as fs / n to obtain m / n sampled audio data, which may be converted by an audio data converter to obtain m converted audio data. Easy to do. In this case, as described in the sixth embodiment, the effects of reducing the capacity of the buffer memory and using a sound board whose sampling frequency is strictly limited can also be obtained.

In the coding according to the fifth to eighth embodiments, since the audio data is substantially thinned out and high-frequency components are removed, the sound quality is degraded accordingly. However, even in such a case, even with a low-performance CPU, band-division-encoded data such as MPEG Audio can be created in real time in software without the need for additional hardware or the like. It can be used as widely used MPEG data. Also, by adjusting the value of the conversion constant, C
A high-performance CPU that can control the degree of thinning and the ratio of high-frequency components to be removed in accordance with the encoding processing performance of the PU
In addition, even a CPU with inadequate performance can perform encoding with sound quality equivalent to its encoding processing capability, and encoding processing can be realized by a CPU with a wide range of performance levels. However, in terms of hardware, the higher the performance of the CPU and the higher the function of the sound board and the higher the data transmission speed in the device, the higher the quality of encoding is possible.

Also, the speech coding of the fifth to eighth embodiments is as follows.
It can be recorded on a recording medium as a voice encoding control program, and can be executed by a personal computer, a workstation, or another device. In the fifth to eighth embodiments, the encoded data is stored in the storage device. However, the encoded data is transmitted to another device via a network or the like.
It is also possible to record or use it in another device. In the fifth to eighth embodiments, the description has been made on the assumption that the processing is performed by the CPU. However, the same applies to the software processing using a DSP instead of the CPU.

Embodiment 9 FIG. The speech coding apparatus according to the ninth embodiment of the present invention controls whether or not to perform data processing for each unit period according to a set constant on sampling data divided into unit periods. Thus, the processing load can be reduced. FIG. 20 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 9 of the present invention. FIG. 21 is a flowchart of speech encoding according to the ninth embodiment, and FIG. 22 is a conceptual diagram for describing speech encoding according to the ninth embodiment. The hardware configuration of the encoding device according to the sixth embodiment is the same as that of the fifth embodiment, and FIG. 11 is used for the description.

As shown in FIG. 20, the speech coding apparatus according to the ninth embodiment includes a speech input unit 901, a register 90
2, input voice sampling unit 903, determination control unit (unit period determination) 904, band division unit 905, coded bit allocation unit 906, quantization unit 907, coding unit 908, coded data recording unit 909, and fixed Sign register 9
10.

[0301] The voice input unit 901 is for inputting voice to be coded. The sound may be input from a microphone as shown in FIG. 11, or may be a line input. The unit period determination constant register 902 is realized by the main memory or the external storage device of FIG. 11, and stores the unit period determination constant. Input audio sampling unit 903
Is implemented by the sound board (input) and the control program in FIG. 11, and performs sampling processing on the sound input by the sound input unit 901. Judgment control unit 904
Determines whether the data sampled by the input audio sampling unit 903 is a coding target period using the constant value stored in the register 902.
The band dividing unit 905 divides the band of the sampling data only when the judgment control unit 904 determines that the period is an encoding target period. The coded bit allocation unit 906 allocates coded bits to the band divided by the band division unit 905. The quantization unit 907 performs a quantization process according to the number of coded bits allocated by the coded bit allocation unit 906. The encoding unit 908 outputs the quantized value output from the quantization unit 907 as encoded audio data. In the ninth embodiment, when the determination control unit 904 determines that the current period is not the encoding target period, the encoding unit 908
The coded data dN corresponding to zero band output stored in a fixed code register 910 described later is output as coded audio data. 904 to 908 are all shown in FIG.
, A main memory, and a program. The encoded data recording unit 909 is realized by the external storage device and the control program in FIG. 11, and records the encoded data output. The fixed sign register 910 is shown in FIG.
1 and stores encoded data dN corresponding to zero band output. The operation at the time of encoding by the speech encoding apparatus according to Embodiment 9 configured as described above will be described below in accordance with the flowchart of FIG. 21 and with reference to FIGS. 20 and 22.

In step 1 of the flow shown in FIG. 21, the audio signal input from the audio input unit 901 is sampled in the input audio sampling unit 903 using the set frequency fs as the sampling frequency. This allows
The sampling data of the frequency fs is output to the determination control unit 904.

At step 2, the judgment control unit 904
It is determined whether or not the sampling data is a coding target period. In this determination, first, a period corresponding to the number p of input audio samples to be processed in one band division is set as a unit period ti, and it is determined whether or not a unit period is a coding target period. The unit period determination constant k used for the determination is a constant that is set in advance by the system as an integer of 1 or more and stored in the register 902. The determination is made with respect to the unit period ti assuming that the encoding target period is established when i = n × k + 1 is satisfied for an arbitrary integer n, and is determined not to be the encoding target period when i is not established.

If it is determined in step 2 that the unit period ti is the encoding target period, steps 3 to 6 are executed to perform the same processing as in the conventional example. That is,
First, in step 3, the band division unit 905 divides the audio data of the unit period ti into M frequency bands. This step is performed in the same manner as in the case of the first conventional example described with reference to FIGS. In step 4, the number of coded bits is allocated to each band in the coded bit allocation unit 906, and the allocation is transmitted to the quantization unit 907. In step 5, the quantization unit 907 quantizes the audio data in the unit period ti for each band divided by the band division unit 905 according to the number of coded bits, and outputs a quantized value. Then, in step 6, the encoding unit 908 sets the quantization unit 9
The encoded audio data is formed and output from the quantized value which is the output of the encoded data 07, and the encoded audio data is recorded in the encoded data recording unit 909.

On the other hand, if it is determined in step 2 that the unit period ti is not the encoding target period, steps 3 to 6
Is not performed, and step 7 is executed following step 2.
In step 7, the coding unit 908 acquires the fixed coded data dN from the fixed code register 910, and outputs this as coded data. Here, the fixed coded data dN is data preset in the fixed code register 910 with the output of each band in the band division being set to zero. The output encoded audio data is recorded in the encoding recording unit 909. As shown in the flow of FIG. 21, the above process is repeated while the input of the audio to be encoded continues, and the encoding is completed immediately after the input of the audio is completed.

The case where the number of input speech samples is p = 32 and the variable constant k is set to 3 is further described with reference to the conceptual diagram of FIG. As shown in the figure, audio data is input, and in the first unit period t1, i = 1 = 0 × 3 + 1.
Holds (n = 0), which is the encoding target period. The sampling data for this unit period is divided into 32 band signals, quantized and encoded, and encoded data d1 is output. In the subsequent unit periods t2 and t3, i = n
There is no integer n that satisfies × k + 1, and it is determined that the current period is not the encoding target period.
The fixed coded data dN is output. The fixed coded data dN is data preset as 32 band signals zero as described above. Thereafter, in the unit period t4, i = 4 = 1 × 3 + 1 holds (n = 4).
Therefore, the sampling data for the unit period is band-divided, quantized, and coded in the same manner as the data at t1, and coded data d4 is output.
Hereinafter, the same processing is performed.

In the speech encoding according to the ninth embodiment, as described above, between the encoded data d1 and d4 based on the input speech, (k−1) pieces of data dN of zero output are included. Data will be obtained. As described in the first conventional example, in the case of MPEG1 Audio layer 1 audio coding, input audio samples are divided into 32 bands using 512 samples before and after the target 32 samples, and divided into 32 bands. Output audio data. Therefore, even if the band output corresponding to the period of 32 samples is encoded as output zero and then decoded and reproduced, the sound is not interrupted at this portion, but is decoded and reproduced together with the encoded data of the band outputs before and after that. . Therefore, since the envelope (temporal change of the sound) of the reproduced sound is continuous, human sound perception does not cause a great deterioration in sound quality.

As described above, in the speech coding apparatus according to the ninth embodiment, the register 902 storing the unit period determination constant k and the sampling data for the unit period are determined based on the unit period determination constant k. A determination control unit 904 that determines whether the data belongs to the encoding target period, and a fixed storage unit that stores fixed encoded data used as a substitute for encoded data obtained by processing data other than the encoding target period. With the provision of the dynamic sign register 910,
1 of sampling data belonging to the encoding target period
/ K processing is performed only on the sampling data of / k, and the processing is not performed on the remaining sampling data that does not belong to the encoding target period. Since dN is output, by setting the value of k, it is possible to reduce the burden on each processing of band division, coded bit allocation, quantization, and coded data generation to a processing amount 1 / k at each stage. Becomes possible.

[0309] Therefore, in the conventional method, even when the real-time encoding processing accompanying the speech input is difficult or impossible due to the lack of performance of the CPU, etc., the load is reduced by setting the variable constant k. The encoding process can be performed in real time. For determining the value of the variable constant k, the CPU used in the apparatus is fixedly assumed, a method of setting based on the encoding processing performance, a value determined in advance for each CPU by simulation, etc. , A method of performing an operation for measuring the encoding processing performance of the CPU prior to the encoding processing, and setting based on the result.

As described above, the sound is not interrupted during reproduction even if there is a period during which the band output is zero, but the longer the period, the greater the deterioration of the sound. It is desirable that the period of zero band output be 32 samples, that is, k = 2. However, if it is necessary to execute the encoding process even if the deterioration of the sound is tolerated, real-time encoding according to the capability of the device is possible by increasing k.

Embodiment 10 FIG. Embodiment 10 of the present invention
Can reduce the processing load by omitting some of the arithmetic processing in band division. FIG. 23 is a block diagram showing a configuration of the speech coding apparatus according to Embodiment 10 of the present invention. As shown in FIG. 23, a speech coding apparatus according to Embodiment 10 includes a speech input unit 1001, an input speech sampling unit 1003,
Band splitting section 1005, coded bit allocating section 100
6, a quantization unit 1007, an encoding unit 1008, an encoded data recording unit 1009, and a register 1011. Further, the apparatus according to the tenth embodiment has the hardware configuration shown in FIG. 11 similarly to the ninth embodiment.

[0312] In the figure, a register 1011 is realized by a main memory or an external storage device, and stores an arithmetic processing determination constant used for arithmetic execution control in band division processing. Also, the band dividing unit 1005 of the tenth embodiment
Includes an arithmetic processing stop unit that obtains the value of the arithmetic processing determination constant from the register 1011 and stops the arithmetic processing in band division. Voice input unit 1001, input voice sampling unit 1003, coded bit allocation unit 1
006, the quantization unit 1007, the encoding unit 1008, and the encoded data recording unit 1009 are the same as those in the ninth embodiment.
1, 903, and 906 to 909, and a description thereof will be omitted.

FIG. 24 is a flowchart of speech encoding performed by the encoding apparatus according to the tenth embodiment, and FIG. 25 is a diagram showing coefficients used in the calculation of the basic low-pass filter processing in band division. Hereinafter, the operation of the speech coding apparatus according to Embodiment 10 will be described with reference to FIG. 24 and with reference to FIGS. 23 and 25.

Step 1 of the flow in FIG. 24 is executed in the same manner as in the ninth embodiment. The audio signal input from the audio input unit 1001 is sampled in the input audio sampling unit 1003, and sampling data is obtained.

Subsequently, in step 2, the band dividing section 1005
Thus, band division processing is performed on the sampling data. As described in the ninth embodiment, MPEG1A
In the audio band division, the audio data for each band is obtained by dividing into 32 bands using 512 samples before and after the target 32 samples, and basic low-pass filtering is performed. In this basic low-pass filter processing, the following equations (1), (2), and (3) are calculated.

[0316]

(Equation 2)

When attention is paid to equation (1), 51
Multiplication processing is performed on two (i = 0 to 511) input audio data Xi by a number Ci obtained from a table. The coefficient Ci is obtained from a coefficient table in which the sample number is compared with the coefficient according to the MPEG audio standard. When the coefficient Ci is graphed, it becomes as shown in FIG. Since the equation (1) is a multiplication process, if the coefficient Ci approaches 0, the product Zi also approaches 0. Further, in equation (2), equation (1)
Is added, the contribution is small for the term whose coefficient Ci is close to 0, and Zi is
It is not necessary to add the values for.

Accordingly, in the equation (1), Zi = 0 for the coefficient Ci close to 0, that is, for the part where i is close to 0 or 511, and Zi = 0. The term corresponding to Zi = 0 is not added,
If the calculation of the band division is performed, the accuracy of the band division is somewhat impaired, but the amount of calculation can be reduced. In this case, as can be seen from FIG. 25, Ci changes in units of 32, and it is desirable that the Ci be determined in units of 32 intervals in which the calculation is terminated. Therefore, the calculation target section for executing the calculation is from i = 32q to i = 32 (8−q) +
255 sections. Here, the arithmetic processing determination constant q is set in advance as an integer satisfying 0 ≦ q ≦ 7 and stored in the register 1011 according to the device performance and the like. By limiting the calculation target section as described above, it is possible to omit the calculation processing by q × １ ／ = q / 8. Therefore, as the calculation processing determination constant q increases, the processing load for the calculation increases. Can be reduced. After band division is performed by omitting some operations as described above in step 2 of the flow of FIG. 24, this band signal becomes a target of processing.
This is the same as step 3 and subsequent steps of the conventional example, and the description is omitted.

As described above, the speech coding apparatus according to the tenth embodiment includes the register 1011 for storing the arithmetic processing determination constant q, and the band division unit 1005 performs the processing in the band division processing based on the arithmetic processing determination constant q. By omitting the arithmetic processing of the basic low-pass filter for some of the samples, the arithmetic processing of the band division unit is reduced by about q / 8 by controlling the value of the arithmetic processing determination constant q. Becomes possible. Therefore, according to the conventional method, even when the real-time encoding process accompanying the voice input is difficult or impossible due to the lack of performance of the CPU, the real-time encoding process can be reduced by setting the arithmetic processing determination constant q. Processing can be performed. Note that the value of the arithmetic processing determination constant q can be determined in the same manner as the determination of the unit period determination constant in the ninth embodiment.

Embodiment 11 FIG. Embodiment 11 of the present invention
Is intended to reduce the processing load by omitting subsequent processing for a part of the band-divided signal. FIG. 26 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 11 of the present invention. FIG.
As shown in FIG. 6, the speech coding apparatus according to the eleventh embodiment includes a speech input unit 1101, an input speech sampling unit 1
103, a band division unit 1105, a coded bit allocation unit 1106, a quantization unit 1107, a coding unit 1108, a coded data recording unit 1109, a register 1112, and a band thinning unit 1118. Further, the device of the eleventh embodiment has the hardware configuration shown in FIG. 11 similarly to the ninth embodiment.

In the figure, the band selection constant register 11
12 is realized by a main memory or an external storage device,
The band selection constant is stored. The band thinning unit 1118 calculates C
A PU is realized by a PU, a main memory, and a program.
Selection extraction is performed based on the band selection constant stored in 112. The voice input unit 1101, the input voice sampling unit 1103, the band division unit 1105, the coded bit allocation unit 1106, the quantization unit 1107, the coding unit 1108, and the coded data recording unit 1109 are the same as those in the ninth embodiment.
01, 903, and 905 to 909, and the description is omitted.

FIG. 27 is a flowchart of speech encoding according to the eleventh embodiment, and FIG. 28 is a conceptual diagram for describing speech encoding according to the eleventh embodiment. The operation of speech encoding according to the eleventh embodiment will be described below with reference to FIG.
Will be described with reference to FIGS. 26 and 28 according to the flowchart of FIG.

Steps 1 and 2 in the flow of FIG. 27 are executed in the same manner as in the ninth embodiment, and the audio signal input from the audio input unit On the other hand, the band dividing unit 1105 divides the signal into M frequency bands, and obtains M band signal data.

In the next step 3, the band thinning section 1
118 obtains the stored band selection constant r from the register 1112, selects and acquires every r band signal data from the M band signal data output from the band division unit 1105, and obtains a total of M / ( (r + 1) band signal data is extracted. r is an integer greater than or equal to 0, which is set in advance according to the device performance and the like and stored in the register 1112.
Here, when r = 2, band signal data marked with a circle every two data is extracted as shown in FIG. The band thinning unit 18 outputs the extracted M / (r + 1) band signal data to the quantization unit 1107.

In step 4, the coded bit allocation unit 1106 determines the number of coded bit allocations for only the band extracted in step 3, based on the band determination constant r obtained from the register 1112. The determined number of coded bits allocated to the M / (r + 1) bands is transmitted from coded bit allocation section 1106 to quantization section 1107. M / (r +
1) The processing is performed on the pieces of data in the same manner as in the first conventional example.

As described above, the speech coding apparatus according to the eleventh embodiment includes the register 1112 for storing the band selection constant r and the band thinning unit 1118, and the band thinning unit 1118 stores the band selection constant r From the M band signal data obtained by the band division processing, M / (r
+1) band signal data is extracted, and the subsequent processing is performed on the extracted data.
And the quantization process can be reduced to about 1 / r. However, the case where r is 1 or more corresponds to this, and when r = 0, the processing load does not change.

Therefore, in the conventional method, even when the real-time encoding processing accompanying the speech input is difficult or impossible due to insufficient performance of the CPU, the load can be reduced by setting the band selection constant r. Real-time processing can be performed. The value of the band selection constant r can be determined in the same manner as the determination of the unit period determination constant in the ninth embodiment.

Embodiment 12 FIG. Embodiment 12 of the present invention
Is capable of monitoring the amount of input data and changing control constants accordingly. FIG.
FIG. 24 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 12 of the present invention. As shown in FIG. 29, the speech coding apparatus according to Embodiment 9 includes
1, register 1202, input audio sampling unit 120
3, determination control unit (unit period determination) 1204, band division unit 1205, coded bit allocation unit 1206, quantization unit 1207, coding unit 1208, coded data recording unit 12
09, fixed sign register 1210, input buffer 12
13, a fixed code register 1210, and an input buffer monitoring unit 1214. That is, an input buffer 1213 and an input buffer monitoring unit 1214 are added to the ninth embodiment. Also, the device of the twelfth embodiment has the hardware configuration shown in FIG. 11, as in the ninth embodiment.

In the figure, the input buffer 1213
It is realized by a memory such as a main memory and temporarily stores data. The input buffer monitoring unit 1214 is realized by a CPU, a main memory, and a program.
In 213, the amount of data held for temporary storage is checked.
This data amount is compared with a preset value, and the value of the constant k of the register 1202 is changed according to the result. The register 1202 is the same as the register 902 of the ninth embodiment except that the value of a constant to be stored is changed by the input buffer monitoring unit 1214. The input audio sampling unit 1203 is the same as the input audio sampling unit 903 of Embodiment 9 except that the input audio sampling unit 1203 outputs the sampled audio data to the input buffer 1213. The determination control unit 1204 is the same as the determination control unit 904 of the ninth embodiment except that the sampled audio data is extracted from the input buffer 1213 and is processed. Also, a voice input unit 1201, a band division unit 1205, a coded bit allocation unit 1206, a quantization unit 1207, a coding unit 120
8, the encoded data recording unit 1209 and the fixed code register 1210 are the same as 901 and 905 to 910 in the ninth embodiment.

FIG. 30 is a flowchart of speech encoding according to the twelfth embodiment. Embodiment 12 below
Will be described with reference to FIG. 29 in accordance with the flowchart of FIG. here,
As the unit period determination constant k, it is assumed that a value “2” predetermined based on the performance of the CPU is stored in the register 2 as an initial value.

In step 1 of the flow of FIG. 30, the audio signal input from the audio input unit 1201 is sampled in the input audio sampling unit 1203 in the same manner as in the ninth embodiment. 1213 and temporarily stored. In step 3, the determination control unit 1204 reads the temporarily stored sampling data from the input buffer 1213.

Step 4 to be performed subsequently will be described later. The determination in step 5 is performed on the data read by the determination control unit 1204 in step 3 in the same manner as in the ninth embodiment. Thereafter, until the encoded data output in step 9 and in step 10, FIG. 4 in Embodiment 9
Steps 5 to 9 and step 10 are omitted since they are executed in the same manner as steps 3 to 6 and step 7 in the flow of No. 1. By executing these steps, according to the setting of the numerical value of the unit period determination constant k, which is 2, the unit periods t1, t3,... ., And the above processing is not performed at t2, t4,.

In the twelfth embodiment, step 3
Is executed, the input buffer monitoring unit 1
214 checks the amount of data in the input buffer 1213,
This data amount is compared with a preset value, and the value of the constant k stored in the register 1202 is changed based on the comparison result. Various methods can be used to control the value of the constant k based on the data amount, but here, the control is performed as follows.

In the initial setting, when the encoding process cannot be performed with the voice input due to an increase in the load on the CPU or the like, the writing to the input buffer 1213 is performed at the same pace. As the processing is delayed, the reading speed for that slows down,
The amount of data stored increases.

Accordingly, the input buffer monitoring unit 1214
If the data amount of the input buffer 1213 exceeds the buffer full level BF which is a preset value, it is determined that the real-time encoding process with the current setting is impossible, and the constant stored in the register 1202 The value of k is increased by 1 and changed to k = 3. In steps 5 to 9 of the flow chart thereafter, for the periods t2 and t3, the band division, the coded bit allocation, and the quantization are not performed, and the coded data dN set in advance is set as the band output 0. Output. When k = 2, which is the first setting, the processing load of these processes is reduced to about に よ り by omitting the processes of band division, coded bit allocation, and quantization once every two times. However, by increasing k to 3, the processing load of these processings is reduced by about 1 by omitting the processing of band division, coded bit allocation, and quantization at a rate of 2 out of 3 times.
/ 3, which can be further reduced. In this manner, the input buffer monitoring unit 1214 changes the value of the constant k to reduce the load on the CPU, and can continue the real-time processing although the sound quality is reduced.

If the amount of data in the input buffer 1213 still exceeds the buffer full level BF in step 4 in the repetition of the flow of FIG. 30, the input buffer monitoring unit 1214 changes the constant k of the register 1202, and Increase to k = 4. Thereby, in steps 5 to 9, with respect to the periods t2 to t4,
The band division, the coded bit allocation, and the quantization are not performed.
By outputting N, the load of band division, coding bit allocation, and quantization processing is reduced to about ４. Thereafter, until the data amount of the input buffer in step 4 becomes equal to or less than the buffer full level BF, the input buffer monitoring unit 1
214 increments the value of k in register 1202.

On the other hand, if the CPU has enough processing power even when the encoding process is performed in accordance with the voice input, such as a decrease in the load on the CPU, the amount of data read from the input buffer 1213 increases. Then, the amount of data to be stored decreases, and eventually, a state where only a small amount of data is stored for a short time continues.

Therefore, in step 4, when the data amount of the input buffer 1213 is lower than the buffer empty level BE which is a preset value, the input buffer monitoring unit 1214 determines that there is enough coding processing capacity. If the value of the constant k is as small as possible, the time of the band output 0 is short, and high-quality encoded data is obtained. Therefore, the input buffer monitoring unit 1214 reduces the value of the constant k by 1 and thereafter, Similarly, in the repetition of the flow of FIG. 30, the value of the constant k of the register 1202 is decreased by 1 until the data amount of the input buffer 1213 becomes equal to or greater than the buffer empty level BE.

In the above method, two values of the buffer full level BF and the buffer empty level BE are used to control the value of the constant k. However, only the buffer full level BF may be used. In this case, for example, if the data amount of the input buffer is equal to a preset buffer full level B
When the value of the constant k is increased until F is reached, and when the voice input and the encoding process are balanced, that is, when the data amount reaches BF, control is performed so as to stop the increase of the constant k. Control can be realized.

As described above, in the speech coder of the twelfth embodiment, the speech coder of the ninth embodiment further includes the input buffer 1213 and the input buffer monitor 1214. The sampling data is temporarily stored in the input buffer 1213 and then read out, and the subsequent processing is performed. Further, the input buffer monitoring unit 1214 checks the amount of data in the input buffer 1213 to determine the unit to be stored in the register. By dynamically controlling the value of the period determination constant k, it is possible to perform a real-time encoding process in accordance with the basic performance of the CPU. It is possible to perform the highest quality speech encoding that can be processed.

Therefore, even when a general-purpose personal computer or the like is used as the audio encoding device to execute the audio encoding process under multitasking, it is possible to cope with a change in the processing capability of the CPU due to execution of another task. hand,
The encoding process can be performed in real time.

Here, the configuration of the speech encoding apparatus is the same as that of the apparatus according to the ninth embodiment except that
And an input buffer monitoring unit 1214 are added, but it is also possible to add an input buffer 1213 and an input buffer monitoring unit 1214 to the device according to the tenth embodiment. By controlling the value of the arithmetic processing determination constant q, it is possible to increase or decrease the arithmetic amount of the basic low-pass filter processing, thereby reducing the processing load and improving the sound quality of the encoded data.

Similarly, the input buffer 1213 and the input buffer monitor 121
4 can be added, and by controlling the value of the band selection constant r, the number of band signal data to be selectively extracted can be increased or decreased to reduce the processing load and the sound quality of the encoded data. It is possible to improve.

Embodiment 13 FIG. Embodiment 13 of the present invention
Is capable of monitoring the amount of coded data to be output and changing the control constants accordingly. FIG. 31 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 13 of the present invention. As shown in the figure, the audio encoding apparatus includes an audio input unit 1301, a register 1302, an input audio sampling unit 1303, a determination control unit 1304, a band division unit 1305, an encoded bit allocation unit 1306, a quantization unit 1307, Encoding unit 130
8, a coded data recording unit 1309, a fixed code register 1310, and a coded data monitoring unit 1315. This configuration is a configuration in which an encoded data monitoring unit 1315 is added to the speech encoding device according to Embodiment 9. Also, the device of the thirteenth embodiment has the hardware configuration shown in FIG. 11 as in the ninth embodiment.

The coded data monitoring unit 1315 includes a CPU,
The amount of encoded data per unit time, which is realized by the main memory and the program and output from the encoding unit 1308, is checked, and the amount of data is compared with a preset value. Change the value of k. The register 1302 is the same as the register 902 of the ninth embodiment except that the value of the stored constant is changed by the encoded data monitoring unit 1315. A voice input unit 1301, an input voice sampling unit 1303,
Decision control section 1304, band division section 1305, coded bit allocation section 1306, quantization section 1307, coding section 1
Reference numeral 308, the encoded data recording unit 1309, and the fixed code register 1310 correspond to 901 and 9 in the fifth embodiment.
Same as 03 to 910. That is, Embodiment 12
The device of the thirteenth embodiment monitors the amount of encoded data and controls the value of a constant stored in a register, whereas the device of the third embodiment monitors the amount of input data.

FIG. 32 is a flowchart of speech encoding according to the thirteenth embodiment. Embodiment 13 below
Will be described with reference to FIG. 31 according to the flowchart of FIG. here,
As the unit period determination constant k, it is assumed that a value “2” predetermined based on the performance of the CPU is stored in the register 2 as an initial value.

Steps 1 to 7 in the flow of FIG. 32 are performed in the same manner as in the ninth embodiment, and therefore, description thereof will be omitted. By performing these steps, 2
According to the setting of the numerical value of the unit period determination constant k, the unit periods t1, t3,...
At 2, t4,..., The above processing is not performed, and encoding with a reduced processing load is executed.

In the thirteenth embodiment, before returning to step 1 and performing repetition, step 8 is executed, and encoded data monitoring section 1315 outputs encoded data amount per unit time output from encoding section 1308. And compare this data amount with a preset value, and according to the result,
The value of the constant k of the register 1302 is changed. Various methods can be adopted to monitor the amount of encoded data and control the value of the constant k. Here, the method is performed according to the following method.

In the initial setting due to an increase in the load on the CPU or the like, if the encoding processing cannot be performed in time, the encoding processing pace is reduced, and the amount of encoded data to be output decreases. Therefore, if the encoded data amount does not reach the encoding minimum level CL which is a preset value in step 8, the encoded data monitoring unit 1315 performs the processing with the input buffer monitoring unit 1214 described in the twelfth embodiment. Similarly,
By increasing the value of the unit period determination constant k stored in the register 2, the load on the CPU is reduced.

Similarly, if the load on the CPU is reduced and there is excess capacity, the amount of encoded data will increase to a certain limit. If it does not fall below the highest coding level CH, the register 130 is used so that high quality coding can be performed.
The value of the unit period determination constant k of 2 is decreased.

In the repetition of the flow of FIG. 32, as in the twelfth embodiment, the above-described increase / decrease in unit period determination effort is repeated, and control is performed so that the constant k becomes an appropriate value. Further, as in the twelfth embodiment, control can be performed by using only the lowest coding level CL without using the two values of the lowest coding level CL and the highest coding level CH.

As described above, in the speech coding apparatus according to the thirteenth embodiment, the apparatus according to the ninth embodiment is further provided with the coded data monitoring section 1315. By examining the amount of encoded data output per unit time, this is used as an index of the encoding processing capability of the CPU at that time,
By dynamically controlling the value of the constant k stored in the register 1302 according to the situation, it is possible for the CPU to perform the highest quality audio encoding that can be encoded at that time. Become. Therefore, similarly to the device of the twelfth embodiment, the CPU at that time by multitasking or the like is used.
Can be changed.

Also, in the thirteenth embodiment, similarly to the twelfth embodiment, the apparatus according to the tenth embodiment or the apparatus according to the eleventh embodiment has the
It is also possible to adopt a configuration in which 315 is added, and a similar effect can be obtained by controlling the value of the arithmetic processing determination constant or the band selection constant.

Embodiment 14 FIG. Embodiment 14 of the present invention
The present invention realizes psycho-aural analysis alternative control by assigning encoded bits, and improves the reproduction sound quality of encoded data without greatly increasing the processing load. FIG. 33 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 14 of the present invention. As shown in the figure, the speech coding apparatus according to the fourteenth embodiment includes a speech input unit 1401, an input speech sampling unit 14
03, band division section 1405, coded bit allocation section 1
406, a quantization unit 1407, an encoding unit 1408, an encoded data recording unit 1409, and a bit allocation control unit (sequential bit allocation) 1416. Also, the device of the fourteenth embodiment has the hardware configuration shown in FIG. 11, as in the ninth embodiment.

In the figure, bit allocation control unit 14
16 is realized by a CPU, a main memory, and a program, and is obtained by division of the band division unit 1405.
The number of bits allocated by the coded bit allocation unit 1406 to the band signal data is calculated according to a predetermined algorithm. Voice input unit 1401, input voice sampling unit 1403, band division unit 1405, coded bit allocation unit 1406, quantization unit 1407, coding unit 1
408 and the encoded data recording unit 1409 are the same as 901, 903, and 905 to 909 of the ninth embodiment, and a description thereof will be omitted.

FIGS. 34 to 36 are flow charts showing the operation of speech encoding according to the fourteenth embodiment. The operation of speech encoding according to the thirteenth embodiment will be described below with reference to FIG.
This will be described with reference to FIG. 33 according to the flowcharts of 4 to 36. Steps 1 and 2 in the flow of FIG.
Is performed in the same manner as in Steps 1 and 2 of FIG. 65 in the conventional example, and band signal data divided into M frequency bands is obtained. According to the MPEG audio standard, it is assumed that M = 32 as in the second conventional example.

Thereafter, in the second conventional example shown in FIG.
Fast Fourier transform and psychological auditory analysis are performed on the band signal data divided into L = 256 to determine the number of coded bits to be allocated. In the fourteenth embodiment, FIG.
In step 3 of the flow of FIG. 4, the bit allocation control unit 16 determines the code for the M = 32 band signal data by the coded bit allocation unit 1406 based on the result calculated by the bit allocation method as a psychological auditory analysis alternative control method. Assignment of coded bits.

First, the total number of bits to be allocated is M
The bit rate is determined from a predetermined bit rate of 256 kbps for layer 1 of PEG audio and 192 kbps for layer 2 of layer 2. Then, the obtained total number of bits is allocated according to the sequential bit allocation method described below.

FIG. 35 is a flowchart showing the procedure of the sequential bit allocation method. Bit allocation control unit 16
Allocates bits to bands 0 to 10 in steps 101 and 103, and then allocates bits to bands 11 to 22 in steps 105 and 107. That is, bits are assigned to the bands 0 to 10 and the bands 11 to 22 twice each. Then step 109
Once in bands 0 to 10 and in step 111 in band 11
Bits are further assigned once to. Then, in step 113, bits are assigned to the bands 23 to 31 once. Determination steps after execution of each of the above steps, steps 102, 104, 106, 108, 110, 11
If it is not determined that the total number of bits has been allocated in any of the steps 2 and 114, the above allocation is repeated from step 101, and the flow ends when the total number of bits is allocated in any of the determination steps.

FIG. 36 is a flowchart showing the procedure for assigning bits to each band. FIG. 36 (a) is used in steps 101, 103 and 109 of the flow of FIG. 35, and FIG. 36 is used in steps 105, 107 and 111 in FIG. 36 (b), and in step 113 of FIG. 35, FIG. 36 (c) is executed.

In the flow of FIG. 36A, first, in step 2
In step 01, the variable a is set to 0, and in step 202, the band 0
Is assigned one bit. If the total number of bits has not been allocated in step 203, a = 0 + 1 = 1 is set in step 204, and the process returns to step 202 according to the determination in step 205, where one bit is allocated to band 1. By repeating this, a = 11 in step 205
36, that is, when it is determined that one bit has been allocated to each of the bands 0 to 10, or when it is determined in step 203 that the total number of bits has been allocated, the flow of FIG. 36A ends. 36 (b) and (c) are the same. When the flow of FIGS. 36 (a) to (c) is completed, the process returns to the original step of the flow of FIG. 35, and when the total number of bits has been allocated, the flow of FIG. Also ends.
Then, step 3 of the flow in FIG. 34 is completed, and steps 4 and 5 in FIG. 34 are executed in the same manner as in the second conventional example.

[0362] The sequential bit allocation method in the apparatus according to the fourteenth embodiment is based on the above-described band 0 to 10 and 11
In this method, bits are allocated to. In the second conventional example, as shown in FIG. 66, these bands have low minimum audible limits, that is, bands that are easy to hear by human hearing, and are divided into 32 bands by band division. The idea is to assign greater weight to bands that are easy for human ears to hear, and to assign bits in order of decreasing weight for each band. Therefore, regardless of whether the band has a large sound pressure or has almost no sound pressure,
Therefore, basically, bits are sequentially allocated as described above regardless of the input signal.

As described above, in the speech coding apparatus according to the fourteenth embodiment, the provision of bit allocation control section 1416 allows bits to be sequentially allocated to bands according to a predetermined algorithm, thereby greatly increasing the processing load. In this manner, code bit allocation can be performed utilizing human auditory characteristics, and encoded data with good reproduced sound quality can be obtained. That is, the speech coding apparatus according to the fourteenth embodiment differs from the apparatus according to the first conventional example in that the bit allocation control unit 1416
, A simple coded bit allocation method according to the simple algorithm shown in FIGS.
2 can improve the reproduction sound quality by performing the processing on the second band, and the processing load is far greater than that of the second conventional example in which the Fourier transform and the psychoacoustic analysis are performed on the 256 band. In speech encoding performed by a general-purpose personal computer, workstation, or the like shown in the hardware configuration of FIG. 21, it is possible to achieve both real-time processing and improved sound quality. The algorithm of the sequential bit allocation method is described in FIGS.
6 is an example, and the present invention is not limited to this. It is possible to execute the same simple sequential allocation even if the order of bands and the number of allocated bits are changed.
Similar effects can be obtained.

Embodiment 15 FIG. Embodiment 15 of the present invention
The audio encoding device according to the present invention realizes psychological auditory analysis alternative control by assigning encoded bits, and further improves the reproduction sound quality of encoded data by considering an output level for each band. . FIG. 37 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 15 of the present invention. As shown in the figure, a speech coding apparatus according to Embodiment 15 includes a speech input unit 1501, an input speech sampling unit 1503, a band division unit 1505, a coded bit allocation unit 1506, a quantization unit 1507, a coding unit 1
508, an encoded data recording unit 1509, and a bit allocation control unit (band output adaptive bit allocation) 1517. This is the same configuration as the device according to the fourteenth embodiment. Also, the device of the fifteenth embodiment has the hardware configuration shown in FIG. 11, as in the ninth embodiment.

In the figure, bit allocation control unit 15
17 is realized by a CPU, a main memory, and a program, and is obtained by division of the band division unit 1505.
The number of bits allocated by the coded bit allocation unit 6 to the band signal data is calculated according to a predetermined algorithm. Such a bit allocation control unit 1517
Embodiment 15 is different from Embodiment 14 except for the function of
This is the same configuration as. Therefore, the audio input unit 1501, the input audio sampling unit 1503, the band division unit 1505,
Encoded bit allocation section 1506, quantization section 1507,
Encoding unit 1508 and encoded data recording unit 1509
Are 901, 903, and 905-9 of the ninth embodiment.
09 and the description is omitted.

FIG. 38 is a flowchart of speech encoding according to the fifteenth embodiment. The following describes the fifteenth embodiment.
Will be described with reference to FIG. 37 in accordance with the flowchart of FIG. 38. Steps 1 and 2 in the flow of FIG. 38 correspond to FIG. 65 in the second conventional example.
Are performed in the same manner as in Steps 1 and 2 above to obtain band signal data divided into M frequency bands. According to the MPEG audio standard, M = 32 as in the second conventional example.
Suppose it was an individual.

Thereafter, in the fourteenth embodiment, in step 3 of the flow of FIG.
16 are sequentially calculated by the bit allocation method. In the fifteenth embodiment, the bit allocation control unit 1517 determines the bit allocation by the band output adaptive bit allocation method. As described above, in the fourteenth embodiment, bits are allocated to the frequency band divided into 32 bands by the band division without considering the sound pressure of the band. In the fifteenth embodiment, bit allocation information for each band is generated based on two factors, that is, whether the band is easily audible to human ears and the sound pressure of each band.

First, the total number of bits to be allocated is obtained from the bit rate of 256 kbps for layer 1, as in the fourteenth embodiment. The bit allocation control unit 17 obtains “band output adaptive weighting (3)” based on “weighting (1) to the band that is easily heard by human ears” and “each band output level ratio (2)”, According to this, the total bit number obtained above is allocated.

(1) Weighting of Bands Easily Recognizable by Human Ear First, "weighting of bands easy to be heard by human ears" is determined as follows based on the minimum audible limit shown in FIG. . (2) Output level ratio of each band Next, an output level ratio for each band, which represents the sound pressure of each band as a ratio, is determined. Here, the following is assumed. (3) Band output adaptive weighting Next, item (1) * item (2) is calculated from the above items (1) and (2) which are two factors to be considered. This gives the following result: (Sb: bandwidth) Based on this result, the bit allocation control unit 17 determines the number of bit allocations by allocating the total number of bits so that the bit allocation of each band approaches the above-mentioned weighting (3). Assign coded bits. Thereafter, steps 4 and 5 in the flow of FIG.
Is executed in the same manner as in the second conventional example.

As described above, in the speech coding apparatus according to the fifteenth embodiment, the provision of the bit allocation control section 1517 makes it possible to determine whether or not the band is audible to human ears and the sound pressure of each band. Bits are allocated to each band based on the two factors described above, so that code bit allocation can be performed utilizing human auditory characteristics without greatly increasing the processing load, and encoded data with good reproduced sound quality can be obtained. .

That is, the speech coding apparatus according to the fifteenth embodiment is different from the first conventional apparatus in that the bit allocation control unit 1
With the configuration to which 517 is added, by executing a simple coded bit allocation method using relatively simple arithmetic processing on 32 bands, it is possible to improve the reproduction sound quality. The processing load is much smaller than that of the second conventional example in which Fourier transform and psychological auditory analysis are performed. It is possible to achieve both time processing and sound quality improvement. Further, the speech coding apparatus according to the fifteenth embodiment has a larger processing load than that of the apparatus according to the fourteenth embodiment due to the processing due to the characteristics of the input speech. Data is obtained.

The calculation method shown in the fifteenth embodiment is an example of the band output adaptive bit allocation method, and the present invention is not limited to this example. Even if it is changed, it is possible to execute the assignment by the same simple arithmetic processing,
Similar effects can be obtained.

Embodiment 16 FIG. Embodiment 16 of the present invention
The speech coding apparatus according to the present invention realizes psychological auditory analysis alternative control by assigning encoded bits, and considers the output level of each band and the number of bits assigned to each band, thereby obtaining encoded data. It is intended to further improve the reproduction sound quality.

FIG. 39 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 16 of the present invention. As shown in the figure, the speech coding apparatus according to the sixteenth embodiment includes a speech input unit 1601, an input speech sampling unit 16
03, band division section 1605, coded bit allocation section 1
606, a quantization unit 1607, an encoding unit 1608, an encoded data recording unit 1609, and a bit allocation control unit (improved band output adaptive bit allocation) 1616. This is the same configuration as the device according to the fourteenth embodiment. Also, the device of the sixteenth embodiment has the hardware configuration shown in FIG. 11, as in the ninth embodiment.

In FIG. 39, bit allocation control unit 1
617 is implemented by a CPU, a main memory, and a program, and calculates the number of bits allocated by the coding bit allocation unit 1606 to the M band signal data obtained by the division by the band division unit 1605 according to a predetermined algorithm. I do. Except for the function of the bit allocation control unit 1617, the device of the sixteenth embodiment has the same configuration as the devices of the fourteenth and fifteenth embodiments. Accordingly, the audio input unit 1601, the input audio sampling unit 1603, the band division unit 1605, the coded bit allocation unit 1606, the quantization unit 1607, and the encoding unit 160
8, and the coded data recording unit 1609 are the same as 901, 903, and 905 to 909 of the ninth embodiment, and a description thereof will be omitted.

FIGS. 40 to 41 are flowcharts of speech encoding according to the sixteenth embodiment. The operation of speech encoding according to the sixteenth embodiment will be described below with reference to FIGS.
1 will be described with reference to FIG.

Steps 1 and 2 of the flow of FIG. 40 are executed in the same manner as steps 1 and 2 of FIG. 65 in the second conventional example, and band signal data divided into M frequency bands is obtained. According to the MPEG audio standard, it is assumed that M = 32 as in the second conventional example.

Then, in the fifteenth embodiment, in step 3 of the flow of FIG.
17 is calculated by the band output adaptive bit allocation method. In the sixteenth embodiment, the bit allocation control unit 161
7 determines bit allocation according to the improved band output adaptive bit allocation scheme. In the fourteenth embodiment, for the frequency band divided into 32 bands by the band division, the number of bits allocated to the band is not considered, and the bit allocation is performed with priority given to the band that is easy for human ears to hear. In the fifteenth embodiment, bits are allocated to each band based on two factors: whether the band is easily audible to human ears and the sound pressure of each band. . In the sixteenth embodiment,
Furthermore, in addition to the two factors in the fifteenth embodiment, the factor of whether the number of bits allocated to each band is sufficient is considered, and the bit allocation information for each band is generated based on the three factors. .

The following describes a bit allocation method used in the sixteenth embodiment. First, 256 for layer 1
Embodiments 14 and 15 determine that the total number of bits to be allocated is determined from the bit rate of kbps.
Is the same as

The bit allocation control unit 1617 includes “weight output for each band (1)”, “weighting to a band that is easy for human ears to hear (2)”, and “weighting corresponding to the number of bits allocated to each band. (3) Allocate the total number of bits obtained based on the above.

(1) Output level of each band First, a scale factor is obtained from the sound pressure of each band. The scale factor takes a value between 0 and 62, and the smaller the value, the higher the sound pressure. Here, it is assumed that the following is performed. (2) Weighting the Band Easily Hearable by Human Ears Next, “weighting the band easy to be heard by human ears” is determined as follows based on the minimum audible limit shown in FIG. Since each band output level is expressed as a scale factor as described above, the meaning is inverted from the weighting value described in the fourteenth embodiment. (3) Weighting corresponding to the number of allocated bits for each band Next, as “weighting corresponding to the number of allocated bits for each band”, based on the minimum audible limit shown in FIG.
Further, it is determined according to the following table created in consideration of not assigning more bits than necessary to the same band.

[0382]

[Table 14]

Assuming that the above factors (1) to (3) exist, the following three items to be considered, item (1)
, (2), and (3), first, item (1) + item (2)
Is calculated. As a result, the next bit allocation information coefficient is obtained. (Sb: bandwidth) The bit allocation control unit 1617 detects a band having the minimum value from the result, and the coded bit allocation unit 6 allocates one bit of the coded bit to the band. If there are a plurality of bands having the minimum value, the lower band is given priority. After that, the bit allocation control unit 161
7 is the item (3) for the band to which the bit is allocated.
, Weighting (+ item (3)) corresponding to the number of bits is performed, and the following result is obtained.

[0384] The above operation by the bit allocating unit 1617 is repeated until the total number of allocable bits is exhausted, and coded bit allocation is performed. , FIG.
The operation in Step 3 of the flow 0 will be described with reference to FIG.

FIG. 41 is a flowchart showing the procedure of the improved band output adaptive bit allocation method. Step 1
At 01, the bit allocation unit 1617 calculates the scale factor of each band as described above. This is a process corresponding to the above (1). In the next step 102, a bit allocation information coefficient for each band is calculated by a weighting process based on the minimum audibility limit. This is a process corresponding to the above (2).

Then, in step 103, a band having the smallest bit allocation information coefficient is detected, and in step 104
Assigns one coded bit to the band. That is, the above items (1) + (2) are performed.

In the next step 105, the above items
Weighting corresponding to (3) is performed, and a weighting coefficient obtained from (Table 14) is added to the band to which the number of bits is allocated in step 104, in accordance with the number of bits currently allocated to the band. . Steps 103 to 10
The operation of No. 5 is repeated until it is determined in step 106 that the assignment of the total number of bits has been completed, and when it is determined that the allocation has been completed, the flow of FIG. 41 ends. Then, step 3 of the flow of FIG. 40 is completed, and steps 4 and 5 of FIG. 40 are executed in the same manner as in the second conventional example.

As described above, in the speech coding apparatus according to the sixteenth embodiment, the provision of the bit allocation control section 1617 makes it possible to determine whether or not the band is audible to human ears and the sound pressure of each band. Since bits are allocated to each band based on three factors of avoiding unnecessary bit allocation to the same band, code bit allocation utilizing human auditory characteristics can be achieved without greatly increasing the processing load. It can be executed and coded data with good reproduction sound quality can be obtained.

That is, the speech coding apparatus according to the sixteenth embodiment differs from the apparatus according to the first conventional example in the bit allocation control unit 1
With the configuration to which 617 is added, by executing a simple coded bit allocation method using relatively simple arithmetic processing on 32 bands, the reproduction sound quality can be improved. In comparison with the second conventional example in which Fourier transform and psychological auditory analysis are performed, the processing load is much smaller. It is possible to achieve both real-time processing and sound quality improvement. Further, the processing load of the speech coding apparatus according to the sixteenth embodiment is larger than that of the apparatus according to the fifteenth embodiment by the amount of processing in consideration of the weight for monitoring the bit allocation status of each band. ,
As a result, encoded data with good reproduction sound quality can be obtained.

The calculation method shown in the sixteenth embodiment is an example of the improved band output adaptive bit allocation method, and is not limited thereto. Even if the weight for the number of allocations is changed, and even if each band output level ratio is used without using the scale factor value, it is possible to execute the allocation by the same simple arithmetic processing, and the same effect is obtained. can get.

Embodiment 17 FIG. Embodiment 17 of the present invention
Is intended to realize a psychological auditory analysis alternative control and to improve the reproduced sound quality of encoded data by assigning encoded bits in consideration of the minimum audibility limit. FIG. 42 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 17 of the present invention. As shown in the figure, the speech coding apparatus according to the seventeenth embodiment includes a speech input unit 1701, an input speech sampling unit 1703, a band division unit 1705, a coded bit allocation unit 1706,
It comprises a quantization section 1707, an encoding section 1708, an encoded data recording section 1709, a bit allocation control section (dynamic bit allocation) 1717, and a minimum audible limit comparison section 1718. This is the same configuration as the device according to the fourteenth embodiment. Also, the device of the seventeenth embodiment has the hardware configuration shown in FIG. 11, as in the ninth embodiment.

In the figure, the minimum audible limit comparing section 171
Numeral 8 is realized by a CPU, a main memory, and a program, compares the M band signal data obtained by the division by the band division unit 1705 with the minimum audible limit, and detects a band smaller than the minimum audible limit. I do. The coded bit allocation unit 1706 does not allocate coded bits to the band detected by the minimum audible limit comparison unit 1718.
Except for the function of the bit allocation control unit 1517, the fifteenth embodiment has the same configuration as the fourteenth embodiment. Therefore, the audio input unit 1701, the input audio sampling unit 1703, the band division unit 1705, the coded bit allocation unit 1706, the quantization unit 1707, and the encoding unit 17
08 and the encoded data recording unit 1709 are the same as 901, 903, and 905 to 909 in the ninth embodiment, and a description thereof will be omitted.

FIG. 43 is a flowchart of speech encoding according to the seventeenth embodiment. The seventeenth embodiment will be described below.
Will be described with reference to FIG. 42 in accordance with the flowchart of FIG. In the apparatus according to the seventeenth embodiment, prior to speech encoding, minimum audibility limit comparison section 1718 uses a memory or the like as internal storage means and sets a minimum audibility limit for M bands (here, 32 bands) as a table. It is something to remember.
This table is to be read and stored from the graph shown in FIG. 66 or a graph of such a graph.

Steps 1 and 2 of the flow of FIG. 43 are executed in the same manner as steps 1 and 2 of FIG. 65 in the second conventional example, and band signal data divided into M frequency bands is obtained. According to the MPEG audio standard, it is assumed that M = 32 as in the second conventional example.

Thereafter, in the second conventional example shown in FIG.
Fast Fourier transform and psychoacoustic analysis including comparison with the minimum audible limit are performed on the band signal data divided into L = 256 to determine the number of coded bits to be allocated. Then, in step 3 of the flow in FIG. 43, the minimum audible limit comparison unit 1718 determines that M = 32
Are compared with the minimum audible limit, and based on the result of the comparison, the coded bit allocation unit 1706 determines
Allocate coded bits to = 32 band signal data.

Then, in the comparison in step 3, the minimum audible limit comparison unit 1718 corresponds to the table stored in advance as described above for the band signal data of M = 32 bands obtained by the division by the band division unit 1705. The band is compared with the minimum audible limit, a band less than the minimum audible limit is extracted, and the result is coded by the coded bit allocating unit 170.
6 is output.

Then, in step 4, the coded bit allocation unit 1706 does not allocate bits to the band less than the minimum audible limit by using the output result of the comparison, and allocates the bit to another band equal to or more than the minimum audible limit. Perform bit allocation to allocate more bits to the band. Steps 4 and 5 are executed in the same manner as in the second conventional example.

As described above, the speech coding apparatus according to the seventeenth embodiment includes minimum audibility limit comparing section 1718, and stores in advance the M bands obtained by dividing by band dividing section 1705. By performing comparison with the minimum audible limit, a band smaller than the minimum audible limit is detected, and the coded bit allocation unit 1706 does not allocate coded bits to the detected band. In addition, it is possible to execute code bit assignment utilizing human auditory characteristics, and obtain encoded data having good reproduced sound quality.

That is, the speech coder of the seventeenth embodiment differs from the apparatus of the first prior art in that the minimum audible limit comparing section 17
With the configuration in which 18 has been added, M =
Since the comparison is made between the 32 band signals and the minimum audible limit, in comparison with the minimum audible limit application in the second conventional example in MPEG1 Audio, the F
FT becomes unnecessary, and the calculation and comparison of the band signal becomes
256 = 1/8, and the processing load can be greatly reduced. Therefore, in the speech encoding performed by a general-purpose personal computer, a workstation, or the like shown in the hardware configuration of FIG. 2, it is possible to achieve both real-time processing and improved sound quality.

[0400] The storage of the minimum audible limit table prior to speech encoding is performed by reading the graph or table shown in Fig. 66.
According to Table D.1 of 1172-3), the minimum audible limit of each band may be obtained and stored. In this table, INDEX is compared with the minimum audible limit. If M = 32 bands, the minimum audible limit is obtained from the table using the value of INDEX close to the center frequency of each of the 32 bands. Can be requested.

Although the ninth to nineteenth embodiments have been described as the embodiments of the present invention, the encoding according to the ninth to thirteenth embodiments substantially eliminates the thinning of the audio data at the band signal data level, Since the filter characteristics are reduced, the sound quality is deteriorated accordingly. However, even in such a case, even with a low-performance CPU, MPEGAud can be implemented in software without additional hardware.
io etc. can be created in real time,
This is called M, which is widely used as an international standard for video coding.
It can be used as PEG data. Also,
By adjusting the value of the variable constant, it is possible to control the thinning degree and the filter characteristics in accordance with the encoding processing performance of the CPU.
However, encoding can be performed with sound quality equivalent to the encoding processing capability, and encoding processing can be realized with a CPU having a wide range of performance levels.

In the coding according to the fourteenth to seventeenth embodiments, the effect of improving the sound quality as compared with the case of performing the psychoacoustic analysis in the second conventional example cannot be obtained. However, the sound quality can be improved as compared with the speech encoding according to the first conventional example which does not perform any psychoacoustic analysis at all, and even in a device such as a general-purpose personal computer or a workstation, it does not require additional hardware or the like. It is possible to improve the reproduction sound quality while performing the real-time encoding accompanying the voice capture.

However, in any of the ninth to seventeenth embodiments, the higher the performance of the CPU and the higher the function of the sound board and the higher the data transmission speed in the device, the higher the quality of the hardware. Encoding is possible. Also, the audio encoding according to the ninth to seventeenth embodiments can be recorded on a recording medium as an audio encoding control program, and can be executed by a personal computer, a workstation, or another device.

In the ninth to seventeenth embodiments, the coded data is stored in the storage device. However, the coded data can be transmitted to another device via a network or the like and recorded or used in another device. It is. Further, Embodiments 9 to
In FIG. 17, it is described that the processing is realized by software processing under CPU control. However, the same applies to software processing controlled by a DSP instead of the CPU.

Embodiment 18 FIG. Embodiment 18 of the present invention
The video and audio coding apparatus according to the present invention, when performing video and audio coding processing on a general-purpose computer by software processing, even if there is an increase in the load on the computer, etc. The encoding process of the video information is stopped using the accumulated amount as an index.

FIG. 44 is a diagram showing a schematic configuration of a video / audio coding apparatus according to Embodiment 18 of the present invention. As shown in the figure, the video encoding device according to the eighteenth embodiment includes a video camera 1801, an audio capture unit 1802, an audio buffering unit 1803, an audio encoding unit 1805, a video capture unit 1806, a video encoding unit 1807, It comprises an encoding load evaluator 1808. As shown in the figure, the video / audio coding apparatus outputs coded audio information and coded video information as device outputs, which are transmitted or recorded as necessary.

[0407] In the same figure, a video camera 1801 is
It takes in video and audio information, and separates and outputs analog audio information and analog video information. Voice capture unit 180
2 inputs analog audio information output from the video camera 1801, and outputs it as digital original audio information composed of discrete digital data. The audio buffering unit 1803 temporarily stores the digital original audio information output from the audio capture unit 1802. The total amount of the original audio information stored in the audio buffering unit 1803 is the original audio buffer amount 1804, which is information used for control in the video and audio encoding device according to the eighteenth embodiment. The audio encoding unit 1805 extracts the original audio information stored in the audio buffering unit 1803, performs compression encoding processing, and outputs encoded audio information. When extracting the temporarily stored original audio information, the audio encoding unit 1803 extracts the earliest (past) original audio information from the temporarily stored original audio information, and stores it in an audio buffer. It is deleted from the ring section 1803. Therefore, the audio buffering unit 1803 performs the FIFO (First
It is desirable to adopt an (In First Out) structure, specifically, it is realized by an algorithm such as a ring buffer.

[0408] The video capture unit 1806 receives the analog video information output from the video camera 1801,
It outputs digital original video information consisting of discrete digital data and a plurality of still images per unit time. Here, the original video information is obtained as having the resolution set in advance. The video capture unit 1806 and the above-described audio capture unit 1802 are realized as a generally equipped video capture board if it is a multimedia-compatible personal computer. The video encoding unit 1807 receives the original video information output from the video capture unit 1806, performs compression encoding, and outputs encoded video information. The encoding load evaluator 1808 evaluates the load in the encoding process of the video / audio encoder, and processes the original video information output from the video capture unit 1806 in the video encoder 1807 in accordance with the evaluation. Control. The control by the coding load evaluator 1808 obtains the coding load evaluation information 1809 by calculation using the coding load reference information 1810, and converts the original video information into a video based on the value of the obtained coding load evaluation information 1809. This is performed by selecting whether to input to the encoding unit 1807 or discard the original video information. When the original video information is discarded, the encoding process of the video encoding unit 1807 is interrupted,
The computer resources (CPU time) of the encoding device are given to the speech encoding unit 1805.

The encoding load evaluator 1808 acquires the encoding load evaluation information 1809 when the original audio buffer amount 18
This is performed by calculating evaluation information using the value of “04” and multiplying the evaluation information by the coding load reference information 1810. In the eighteenth embodiment, when calculating the evaluation information, if the original audio buffer amount 1804 becomes half or more of the storable amount in the audio buffering unit 1803,
The evaluation information is set to 0%.
00%.

[0410] The coding load reference information 110 is information indicating the reference of the processing amount of the video coding processing, and is set as, for example, "amount indicating how much video processing is performed when the audio buffer is empty". It should be noted that, in the eighteenth embodiment, when the evaluation information is multiplied by the coding load reference information 1810, the evaluation information is used as it is as the value of “1”. 180
9 is set. Accordingly, the coding load evaluation information 109 is 0% or 100%. In the case of 100%, the coding load evaluation unit 108 inputs the original video information input at that time to the 100% video coding unit 1807. And
In the case of 0%, all the original video information is discarded. Then, the processing of the video encoding unit 1807 is interrupted, and the computer resources (CP
(U time) is passed to the speech encoding unit 1805.

[0411] Embodiment 18 thus configured
The outline of the operation of the video / audio coding apparatus according to the above is as follows. That is, in the video / audio coding apparatus,
When the analog audio information output from the video camera 1801 is input, the audio capture unit 1802 outputs the digital audio information as digital original audio information including discrete digital data. The digital original audio information output from the audio capture unit 1802 is
Is temporarily stored. Then, the speech encoding unit 1805
Extracts the earliest (past) original audio from among the original audios stored in the audio buffering unit 1803, and extracts the extracted original audio into the audio buffering unit 1803.
3, and compresses and encodes the original audio to output as encoded audio information. The audio encoding unit 1805 updates the original audio buffer amount 1804 which is a value indicating the total amount of the original audio stored in the audio buffering unit 1803, and the original audio buffer amount 1804 is encoded by the video / audio encoding device. It is stored as information for processing.

[0412] When analog video information output from the video camera 1801 is input, the video capture unit 1806 includes a plurality of still images per unit time, which are composed of discrete digital data and have a predefined resolution. It is output as digital original video information composed of information. When the original video information output from the video capture unit 1806 is input, the video encoding unit 1807 performs compression encoding and outputs the encoded information. Then, the encoding load evaluator 1808 calculates the encoding load evaluation information 1809, and inputs the original video information to the video encoding unit 1807 or outputs the original video information according to the calculated value of the encoding load evaluation information 1809. The information is discarded, the processing of the video encoding unit 1807 is interrupted, and the computer resources (CPU time) are
An operation to determine whether to surrender to 805 or not is determined. When the original video information is input, the video encoding unit 1807 performs a compression encoding process on the original video information, and outputs encoded video information.

[0413] Fig. 45 is a diagram schematically illustrating an operation of capturing and encoding a certain video and audio in the video and audio encoding device according to the eighteenth embodiment. Here, the present apparatus is realized by a general-purpose computer such as a personal computer, and the general-purpose computer is operated by a multitask operation system capable of executing a plurality of tasks (tasks) in parallel. It is assumed that the video / audio coding process is handled as video coding and audio coding tasks on the operation system. For each task including video encoding and audio encoding, CPU time, which is a computer resource, is assigned by the operating system, and during the assigned period, each process can be executed under the control of the CPU. Here, it is assumed that when each task finishes a series of processing and releases the allocated computer resources (CPU time), the operation system is controlled to allocate to another task.

[0414] In the figure, it is assumed that time progresses from the top to the bottom, and the squares in the figure indicate that each process (task) on the multitask operation system consumes computer resources (CPU time). It indicates that. Dashed arrows connecting the squares indicate that the process has switched. The dashed arrow is a diagonal line, and the angle of the diagonal line indicates the time required for the process switch, that is, the overhead for task switching of the multitask operation system. In the following description, this overhead is assumed to be relatively small in comparison with the processing in each task, and is ignored in the description.

[0415] In the figure, the column indicated by "video encoding process" indicates the time consumed by the process for encoding the video information.
The operation time of the process of executing the process 08 and the process of the video encoding unit 1807 is shown. Further, a column indicated by “speech encoding process” indicates a time consumed by a process for encoding speech information. Is shown. Further, the “other processes” indicate the working time of any process other than the “video encoding process” and the “audio encoding process”. The “source audio buffer amount” is the original audio buffer amount 1804 at that time.
This is shown as a ratio to the maximum buffer amount (the amount that can be stored in the audio buffering unit 1803).

[0416] In the configuration of the video / audio coding apparatus, the video camera 1801 is a peripheral device that functions relatively independently by being connected to a general-purpose computer that realizes the apparatus. , C
The operation is performed substantially in parallel with the process executed by the PU control. Also, the voice capture unit 180
Similarly, the video capture board that implements the video capture unit 1806 can also function relatively independently, and the audio capture unit 1802 and the video capture unit 1806 are also generally parallel to the above processes. The operation is performed.

That is, even when a video or audio encoding process and other processes are being executed, video and audio are fetched substantially in parallel, and original video information and original audio information are created by digitization. Then, the original audio information is stored in the audio buffering means 1803.

The following is a description of the eighteenth embodiment of this example.
Will be described with reference to FIGS. 44 and 45. First, the video camera 18
01 takes in the video / audio information and outputs it separately into analog audio information and analog video information. The analog video information is input to the video capture unit 1806, and the video capture unit 1806 outputs digital original video information including a plurality of pieces of still image information by analog / digital conversion processing as described above. Since this process is mainly performed by the operation of the video camera 1801 and the process executed by the capture board as the video capture unit 1806, as shown in FIG.
It is processed substantially in parallel with each process on the operating system that consumes CPU time.

[0419] The encoding load evaluator 1808 inputs the once output original video information, and checks the original audio buffer amount 1804 at that time. Here, it is assumed that the audio has not yet been input to the audio buffering unit 1803, and the original audio buffer amount 1804 is 0%. Therefore, the value is lower than the predetermined reference value of 50%. As described above, in the present embodiment, the coding load reference information 1810 may be set to “1” and may not be considered in the multiplication. No coding load evaluation criterion information 1
809 is 100% as it is with the evaluation information. Therefore,
The encoding load evaluator 1808 calculates 1 of the input original video information.
00% is output to the video encoding unit 1807. The video encoding unit 1807 performs a video encoding process on the original video information, and releases the CPU time when the process is completed. An encoding load evaluation unit 1808 and a video encoding unit 1807
The above processing corresponds to the part A which is the video encoding process in FIG.

[0420] On the other hand, the audio capture unit 1802 receives the analog audio information output from the video camera 1801, and outputs it as digital original audio information by analog / digital conversion processing. The original audio information is input to the audio buffering unit and temporarily stored, and the audio buffering unit 1803 updates the original audio buffer amount 1804 held by the video / audio encoding device according to the amount of storage. Since this process is mainly performed by the operation of the video camera 1801 and the process executed by the capture board as the audio capture unit 1802, the process is performed substantially in parallel with each process on the operating system as shown in FIG. You. Here, it is assumed that this process is executed in parallel with the video encoding process A, and the original audio buffer amount has reached 30%.

[0421] The audio encoding unit 1805 reads out a fixed amount (original audio readout amount) of the original audio information from the audio buffering unit 1803 from the previously (past) accumulated amount of original audio information, and converts the read-out original audio information into audio. Buffering unit 18
03, and the original audio buffer amount 1804 is updated. Further, the audio encoding unit 1805 encodes the original audio information. In the eighteenth embodiment, the original audio readout amount is set to 30% of the maximum buffer amount.
Since all the data has been stored, the entire data is read and encoded, and the CPU time is released when the encoding is completed. This processing by the audio encoding unit 1805 corresponds to part B which is the audio encoding process in FIG.

Here, as shown in FIG. 45, since the “other application” was accidentally activated and requested the CPU time, the “other application” consumes the CPU time. The “other application” has a relatively large processing load, and occupies the CPU time for a while and then releases it. This step (process) relating to the processing of another operation corresponds to a part C which is another process in FIG. It is assumed that the processes by the video camera 1801, the audio capture unit 1802, and the video capture unit 1806 are being executed in parallel with the process of the part C. Therefore, the original audio information is temporarily stored, and the original audio buffer amount reaches 60% as shown in FIG.

Next, the encoding load evaluator 1808 again sends C
Although the allocation of PU time has come around, the 60% of the original audio buffer amount 1804 at this point is 50% or more of the reference value, so the evaluation information obtained by the coding load evaluator 1808 is 0. %, And the encoded load reference information 1
Even if multiplied by 10 “1”, the coding load evaluation information 180
9 is 0%. Therefore, the encoding load evaluator 1808
Discards the original video information at this time, and
07 is not performed, and the CPU time is released immediately. This processing of the encoding load evaluator 1808 corresponds to the part D in FIG.

[0424] The speech encoding unit 1805 has a CPU
Since the time has been allocated, the audio buffering unit 1
30% of the original audio information is read from 803, and the corresponding original audio information is deleted from the audio buffering unit 1803;
The original audio buffer amount 1804 is updated. The original audio buffer amount is reduced from 60% to 30%. Further, the audio encoding unit 1
Reference numeral 805 encodes the original audio information, and releases the CPU time when the encoding is completed. This processing by the audio encoding unit 1805 corresponds to the part E in FIG.

[0425] CPU time is allocated to the encoding load evaluator 1808. At this point, the original audio buffer amount 1804
Is 30%, which is less than 50% of the reference value.
5, the encoding load evaluation information 1809 becomes 100%, and the video encoding unit 1807
In, encoding of original video information is performed. Since the original video information is more complicated than in the previous video coding process, the coding process takes longer than in the case of the process A part, and the video coding process takes a relatively large amount of CPU time. The conversion process frees up CPU time. The above processing performed by the encoding load evaluation unit 1808 and the video encoding unit 1807 corresponds to the F portion of the video encoding process in FIG. In parallel with this process F, the original audio information is accumulated, and the original audio buffer amount reaches 90%.

[0426] Since the CPU time has been allocated to the audio encoding unit 1805, the audio encoding unit 1805 reads out 30% of the original audio information from the audio buffering unit 1803, and converts the original audio information to that amount. Buffering unit 18
03, and the original audio buffer amount 1804 is updated. The original audio buffer amount 1804 changes from 90% to 60%. Further, the audio encoding unit 1805 encodes the original audio information, and releases the CPU time when the encoding is completed. This processing by the audio encoding unit 1805 corresponds to the G section in FIG.

[0427] CPU time is allocated to the encoding load evaluator 1808. At this point, the original audio buffer amount 1804 has reached 90%, which is 50% or more of the reference value.
As in the case of the above process D, the coding load evaluation information 1809 is 0%, and the coding load evaluation unit 1808
The original video information at this point is discarded, and the video encoder 1807
Video encoding processing is not performed in the
Time is released. This processing of the encoding load evaluator 1808 corresponds to the portion H in FIG.

[0428] Since CPU time has been allocated to the audio encoding unit 1805, the audio encoding unit 1805 reads 30% of the original audio information from the audio buffering unit 1803, and substitutes the corresponding original audio information with the audio buffering unit. 180
3 and the original audio buffer amount 1804 is updated.
Further, the audio encoding unit 1805 encodes the original audio information, and releases the CPU time when the encoding is completed. This processing by the audio encoding unit 1805 corresponds to the I part in FIG.

In FIG. 44, while the video and audio are being captured by the video camera 1801, the video encoding and audio encoding processes are executed as described above, so that the video associated with the capturing is obtained. Voice coding is performed. Then, after the capturing of the video and audio ends, the encoding ends.

FIG. 46 is a diagram for illustrating such a coding operation in the video / audio coding apparatus according to the eighteenth embodiment over a longer period of time. In the figure, in the section A, the audio and video encoding processes are performed in a well-balanced manner, and the audio buffer amount keeps a value equal to or less than the reference value. Thereafter, as shown in FIG. Since the time is monopolized, the original audio information is excessively accumulated in parallel. So the following B
In the section, audio encoding is preferentially processed in order to process the stored original audio information. After the original audio information decreases and becomes equal to or less than the reference value, normal processing is executed again as seen in the section C in FIG.

As described above, according to the video and audio encoding apparatus of the eighteenth embodiment, audio buffering section 1803
And an encoding load estimating unit 1808, wherein the encoding load estimating unit 1808 to which the original video information to be encoded with respect to the video has been input is output by the video encoding unit 180 before the encoding process. Audio buffering section 180
The original audio buffer amount 1804, which is the amount of the unprocessed audio information stored in No. 3, is checked. If it is sufficiently small, video encoding is performed. Since the video information is discarded and video encoding is not performed and the CPU time is transferred to the audio encoding unit,
The effect of lack of computer resources due to consumption by other applications or the video encoding unit itself is limited to dropped frames of video that are not easily recognized as a problem, and audio is interrupted due to video encoding. Can be avoided.

According to the video and audio coding apparatus of the eighteenth embodiment, the function of video coding section 1807 can be coded when coding load evaluating section 1808 outputs video information. Things. As long as it has a single function of encoding the input video, it can be applied as the video encoding unit 1807 of the encoding device according to the eighteenth embodiment. That is, an existing video encoding unit such as an image compression subroutine can be applied without any internal change. This means that in a video information operation environment on a general-purpose computer that has a configuration in which the compression subroutine is modularized and can be added on later, it is possible to apply the subroutine as it is and use it for video / audio encoding. This has the special effect that software development can be performed efficiently.

In the eighteenth embodiment, each process that performs multitasking is switched by the process itself releasing computer resources (CPU time). However, the present invention is not limited to this mode. It is not limited. For example, the multitasking operating system may give a certain amount of CPU time to each process, and switch to another process unconditionally when each process runs out of the CPU time. In this case, if the video encoding process monitors the progress of the audio encoding process before using up the CPU time, and if necessary, voluntarily releases the CPU time, the computer resources can be more effectively used. (CPU time) can be assigned so that a good encoding result can be obtained.

In the eighteenth embodiment, video information (still image information) to be encoded is discarded,
This frees up CPU time for the video encoding process. That is, when the still image information is input as “still image at 0 second point, still image at 1 second point, still image at 2 second point”, if necessary, the still image at 1 second point is dropped, Still image at 0 second point, still image at 2 second point ". However, the encoded information that is finally output does not necessarily have to have a small number of frames. That is, a time stamp indicating which point (time) the still image information is in is inserted in the still image information, and the video encoding unit checks the time stamp to determine whether or not there is a frame drop. Is recognized, and if a frame is dropped, an image corresponding to the frame (the same image as the previous still image, a code indicating the same image as the previous image, etc.) is output, and finally, The coded video information to be output is complete in terms of face value and without dropping frames. With this method, MPEG
Even when outputting video information for which the number of frames of video information must be guaranteed to a predetermined value (such as 30 frames per second), such as the (Motion Picture Experts Group) standard, it can be easily handled.

Embodiment 19 FIG. Embodiment 19 of the Invention
The video / audio coding apparatus according to the present invention prevents the interruption of the voice even when the load increases in the software processing in the general-purpose computer or the like, as in the eighteenth embodiment. It controls the prediction process used for encoding.

FIG. 47 is a diagram showing a schematic configuration of a video / audio coding apparatus according to Embodiment 19 of the present invention. As shown, the video encoding apparatus according to the nineteenth embodiment includes a video camera 1901, an audio capture unit 1902, an audio buffering unit 1903, an audio encoding unit 1905, a video capture unit 1906, a video encoding unit 1923, The video encoding unit 1923 includes an encoding load evaluation unit 1921, and includes an inter-frame prediction processing unit 1924 and a frame encoding unit 1925. Also, the encoded audio information and the encoded video information are output as the device output, as in the eighteenth embodiment.

In the figure, the coding load evaluator 1921
Obtains the encoding load evaluation information 1922 by calculation based on the original audio buffer amount 1904 and the encoding load reference information 1910. An inter-frame prediction processing unit 1924 included in the video encoding unit 1923 obtains a motion vector between still image information pieces in order to reduce temporal redundancy of video and perform compression encoding, and performs prediction coding with motion compensation. The motion vector is output for conversion. Video encoding unit 1923
Are encoded using the motion vector output by the inter-frame prediction processing unit 1924, and output as encoded video information.

The video camera 1901, audio capture unit 1902, audio buffering unit 1903, audio encoding unit 1905, and video capture unit 1906 are the same as those of the eighteenth embodiment.
And 1806, and the description is omitted.

When the coding load evaluation information 1922 is calculated by the coding load evaluation unit 1921, the audio buffering unit 193 calculated from the original audio buffer amount 104 is used.
Is obtained by multiplying the evaluation information indicating the ratio of the free space of the buffer and the encoding load reference information 110 by using both. The coding load reference information 1910 is the same as that in the eighteenth embodiment, and
9 is also fixedly set to "1".
Therefore, in the nineteenth embodiment, the encoding load reference information 1
No. 910 does not need to be taken into consideration, and the ratio of the free space of the buffer of the audio buffering unit 193 obtained as the evaluation information is used as it is as the encoding load evaluation information 1922. If the buffer is empty, The value is 100%, and 0% when full.

[0440] In general, in compression encoding, intra-frame encoding for compressing a still image of one frame (corresponding to one screen) based on the spatial correlation between the still image and one frame (e.g., consecutive frames) For still images, there is inter-frame coding that performs compression based on the temporal correlation, and intra-frame coding is fundamental. However, when these two are combined, encoded data with a high compression rate can be obtained. Will be obtained. In order to perform inter-frame coding, a motion for each frame is detected as a motion vector, a predicted image is generated with motion compensation using the motion vector, and the predicted image is encoded with the image to be encoded. A technique of compressing the difference data is used.

[0441] The inter-frame prediction processing unit 1924 obtains a motion vector between still image information used for a predicted image generation process for compressing and encoding by reducing temporal redundancy of a video. In the nineteenth embodiment, the inter-frame prediction processing unit 1924 performs prediction processing by a specified ratio. That is, the encoding load evaluation information 1
922 is input, and an inter-frame prediction process is performed on the initial value by a ratio indicated by the coding load evaluation information 1922. If the coding load evaluation information 1922 is 100%, an inter-frame prediction process is performed by a maximum amount which is an initial value, and the obtained optimal motion vector is output. On the other hand, if the coding load evaluation information 1922 is 50%, the inter-frame prediction processing is performed by an amount of 50% of the initial value, and the optimum motion vector obtained at that time is output. In any case, the frame encoding unit 1925 performs an encoding process using the output motion vector. In the process of obtaining a motion vector, an optimum motion vector can be obtained as much as the amount of processing is increased, so that the difference between the predicted image and the encoding target image is reduced. You can do it. On the other hand, if the amount of processing is reduced, an optimal vector cannot be obtained, and the compression ratio decreases. Note that maintaining the compression ratio without increasing the processing amount is possible if image quality is sacrificed.

The twelfth embodiment thus constituted
The outline of the operation of the video / audio coding apparatus according to the above is as follows. That is, in the video / audio coding apparatus,
The encoding load evaluator 1921 includes the audio buffering unit 1
903 stores the original audio information, and outputs the encoding load evaluation information 1922 according to the original audio buffer amount 1904 at the time when the original audio buffer amount 1904 is updated according to the accumulated amount and the encoding load reference information 1910. . Further, the video encoding unit 1923 encodes and outputs the original video information output from the video capture unit 196. At this time,
The inter-frame prediction processing unit 1924 obtains a motion vector between still image information and performs encoding using the motion information in order to reduce temporal redundancy of video and perform compression encoding. Accordingly, the frame encoding unit 1925 performs encoding using the motion vector output from the inter-frame prediction processing unit 1924, and outputs the encoded video information. Reading and encoding of the original audio information are performed in the same manner as in the eighteenth embodiment.

[0443] The operation of an example of encoding processing for a video and audio by the video and audio encoding apparatus according to the nineteenth embodiment will be described below. Here, Embodiment 1
8, the video / audio encoding process includes video encoding (processing of the encoding load evaluator 1921 and the video encoder 1923) and audio encoding (the audio encoder 1905) according to the control of the operating system in the general-purpose computer. Processing)
When each task to which CPU time is allocated executes a series of processing and releases computer resources (CPU time), the operating system gives the other tasks the CPU time. The assignment is controlled.

First, as in the eighteenth embodiment, the video camera 1901 fetches video / audio information, and outputs it separately into analog audio information and analog video information. And
The audio capture unit 1902 receives the analog audio information output from the video camera 1901 and outputs it as digital original audio information. Audio buffering unit 1903
Stores the original audio information and updates the original audio buffer amount 1904 according to the accumulated amount. Meanwhile, the video capture unit 19
Reference numeral 06 inputs analog video information output from the video camera 1901 and outputs it as digital original video information.

[0445] The coding load evaluator 1921 checks the original audio buffer amount 104 at that time. Here, since about 30% of the input original audio information has been stored in the buffer, the coding load evaluation information 1922 is set to 70%.
The inter-frame prediction processing unit 1924 performs encoding load evaluation information 1
922 is obtained. And encoding load evaluation information 1922
Is 70%, so that the inter-frame prediction process is performed only for 70% of the initial value, an optimal motion vector is obtained from the inter-frame prediction process, and this is output to the frame encoding unit 1925. The frame encoding unit 1925 encodes the video information using the motion vector, outputs the encoded video information as encoded video information, and releases the CPU time allocated to the video encoding.

[0446] The audio encoding unit 1905 reads a fixed amount (original audio readout amount) of the original audio information from the audio buffering unit 1903 from the previously (past) accumulated original audio information, and converts the read original audio information into audio. Buffering unit 19
03 and the original audio buffer amount 1904 is updated. Further, the audio encoding unit 1905 encodes the original audio information. In the eighteenth embodiment, the original audio readout amount is set to 30% of the maximum buffer amount.
Since all the data has been stored, the entire data is read and encoded, and the CPU time is released when the encoding is completed.

The coding load evaluator 1921 to which the CPU time has been assigned confirms the original audio buffer amount 104 at that time. Immediately after the above audio encoding processing, and the audio stored in the original audio buffer amount 104 is 0%, the encoding load evaluator 1921 sets the encoding load evaluation information 1922 to 100%. Then, the inter-frame prediction processing unit 1924 sets the encoding load evaluation information 1922 to 10
Since it is 0%, an inter-frame prediction process is performed by the maximum amount that is an initial value to obtain an optimal motion vector, and the frame encoding unit 1925 encodes video information using the motion vector, and performs encoding. The video information is output, and when the encoding is completed, the CPU time is released.

In FIG. 47, while video and audio are being captured from the video camera 1901, the video encoding and audio encoding processes are executed as described above, so that the video associated with the capturing is processed. Voice coding is performed. And after the end of the video and audio capture,
The encoding also ends.

As described above, according to the video and audio coding apparatus of the nineteenth embodiment, the audio buffering section 1903
And an encoding load estimating unit 1921, an inter-frame prediction processing unit 1924, and a video encoding unit 1923 including the frame encoding unit 1925.
21 is the amount of unprocessed audio information stored in the buffer at that time before the video encoding unit 1923 performs encoding;
That is, by confirming the original audio buffer amount 1904 and instructing the amount of processing in the inter-frame prediction processing unit 1924 according to the amount, the CPU time consumed in video encoding is controlled. The effect of lack of computer resources due to the consumption of the encoding unit itself was limited to a temporary decrease in compression rate and image quality, which is not easily recognized as a problem, and audio was interrupted due to video encoding. You can avoid falling.

In the video / audio coding apparatus according to the nineteenth embodiment, the function of video coding section 1923 is as follows.
Is output, processing is performed in response to this. Therefore, it is necessary that the input of the coding load evaluation information 1922 and the processing corresponding to the coding load evaluation information 1922 be performed. As in the eighteenth embodiment, the modularized subroutine cannot be directly applied. However, when reducing the load of video encoding, video information is not discarded as in the eighteenth embodiment, but the amount of processing is reduced. Compared with the eighteenth mode, this embodiment has an effect that coded video information with a smooth motion can be obtained.

In the nineteenth embodiment, the CPU time consumed in video encoding is controlled by using the inter-frame prediction process, that is, by adjusting the amount of calculation for finding an optimal motion vector. The present invention is not limited to such a system. For example, a method of simplifying other video coding processes, such as omitting a part of the color information coding process, may be adopted.

Also, in the nineteenth embodiment, the ratio of the free space of the buffer obtained from the original audio buffer amount is used as the encoded additional evaluation information as it is, but other evaluation methods can be used. For example, an evaluation method is used in which the coded additional evaluation information is set to 100% until the original audio buffer amount exceeds a certain value, but is reduced to 50% and 30% after exceeding the certain value. You can also.

Embodiment 20 FIG. Embodiment 20 of the present invention
The video / audio coding apparatus according to the present invention prevents the interruption of the voice even when the load increases in the software processing in the general-purpose computer or the like, as in the eighteenth embodiment. It changes the video resolution used for encoding.

FIG. 48 is a diagram showing a schematic configuration of a video / audio coding apparatus according to Embodiment 20 of the present invention. As shown in the figure, the video encoding device according to Embodiment 20 includes a video camera 2001, an audio capture unit 2002, an audio buffering unit 2003, an audio encoding unit 2005, a video capture unit 2031, a video encoding unit 2035, The image encoding unit 2035 includes an encoding load evaluation unit 2032, and includes a video encoding unit main body 2036 and a resolution correction information adding unit 2037. Also, the encoded audio information and the encoded video information are output as the device output, as in the eighteenth embodiment.

[0455] In the figure, the video capture unit 2031
Creates digital original video information composed of a plurality of still images from analog video information in the same manner as in the eighteenth and nineteenth embodiments.
At 0, video resolution information 2034 to be described later is input,
The still image information is created as having a resolution corresponding to the input video resolution information resolution. The video capture unit 2031 is realized by a video capture board, as in the eighteenth embodiment. Here, it is assumed that the board can specify a resolution. The coding load evaluation unit 2032 calculates the coding load evaluation information 2033, and calculates the calculated coding load evaluation information 2033.
The video resolution information 2034 is output according to the value of 33. The video encoding unit 2035 includes a video encoding unit main body 2036 described later and a resolution correction information adding unit 2037, encodes original video information output from the video capture unit 2031, and outputs encoded video information. . The video encoding unit main body 2036 is included in the video encoding unit 2035, and performs an actual video encoding process. Resolution correction information adding unit 2
Reference numeral 037 denotes a coded video that is included in the video coding unit 2035, adds resolution information to the internal coded video information output by the video coding unit main body 2036, and becomes the device output of the video / audio coding device. Create information.

A video camera 2001, an audio capture unit 2002, an audio buffering unit 2003, and an audio encoding unit 2005 are the same as those in the eighteenth embodiment.
1 to 1803 and 1805, and the description is omitted.

In the calculation of the coding load evaluation information 2033 by the coding load evaluation section 2032, basic evaluation information is calculated based on the value of the original audio buffer amount 2004, and Multiply by the value. In the twentieth embodiment, when calculating the evaluation information, the evaluation information is set to 0% if the original audio buffer amount 2004 is more than half of the original audio storage capacity of the audio buffering unit 2003, and if less than half. , 100%
It is assumed that. Also, the encoding load reference information 2010
Is the same as in the eighteenth embodiment, and is also fixed to “1” in the twentieth embodiment. Therefore, the evaluation information becomes the encoding load evaluation information 2033 as it is, and the encoding load evaluation information 2033 is set to 0.
% Or 100%. Coding load evaluator 203
2 generates video resolution information 2034 using the coding load evaluation information 2033 and outputs this to the video coding unit 2035. At this time, if the encoding load evaluation information 2033 is 100%, the video resolution information 2034 indicates “320 pixels in width and 240 pixels in height”. 160 pixels, height 120 pixels ".

In the twentieth embodiment, “320 pixels wide and 240 pixels high” are set as the initial values of the video resolution information 2034, but the original audio buffer amount 2004 is If it exceeds 50%, the coding load evaluator 2032 as described above
In the calculation by, "width 160 pixels, height 120
To "pixels".

The twentieth embodiment thus constituted
The outline of the operation of the video / audio coding apparatus according to the above is as follows. That is, in the video / audio coding apparatus,
The video capture unit 2031 includes video resolution information 2034
Is input, digital original video information composed of still image information having the resolution is created by converting the input analog video information, and output. Further, the coding load evaluation unit 2032 calculates the coding load evaluation information 2033, and outputs the video resolution information 2034 according to the calculated value of the coding load evaluation information 2033. Then, the video encoding unit 2035 encodes and outputs the original video information output from the video capture unit 2031. At this time,
The video encoding unit main body 2036 performs an actual video encoding process. Accordingly, the resolution correction information adding unit 2037
Adds resolution information to the encoded video information output by the video encoding unit main body 305. For audio handling,
This is the same as Embodiment 18.

[0460] The operation of an example of encoding processing for a video and audio by the video and audio encoding apparatus according to the twentieth embodiment will be described below. Here, Embodiment 1
8, the video / audio encoding process includes video encoding (processing of the encoding load evaluation unit 2032 and the video encoding unit 2035) and audio encoding (the audio encoding unit 2005) in accordance with the control of the operating system in the general-purpose computer. Processing)
When each task to which CPU time is allocated executes a series of processing and releases computer resources (CPU time), the operating system gives the other tasks the CPU time. The assignment is controlled.

First, as in the eighteenth embodiment, the video camera 2001 takes in video / audio information, and outputs it separately into analog audio information and analog video information. And
The audio capture unit 2002 receives the analog audio information output from the video camera 2001 and outputs it as digital original audio information. Audio buffering unit 2003
Accumulates the original audio information and updates the original audio buffer amount 2004 according to the accumulated amount.

The coding load evaluator 2032 checks the original audio buffer amount 2004 at that time. About 30% of the input original audio information is stored in the buffer, which is lower than a predetermined reference value of 50%.
The evaluation information is 100%. Then, the coding load reference information 2010 having the value “1” as described above does not affect the result even if the multiplication processing is performed, and thus the coding load evaluation information becomes 100%. Therefore, the video resolution information 2034
Is “320 pixels wide and 240 pixels high”, and the encoding load evaluation unit 2032 outputs the video resolution information 2034 to the video capture unit 2031 and the video encoding unit 2035.

On the other hand, the video capture unit 2031 receives the analog video information output from the video camera 2001 and outputs it as digital original video information. At this time, the video capture unit 2031 sets the encoding load evaluation unit 2035
Before the video resolution information 2034 is input, and “width 320” which is the initial value of the video resolution information 2034
The pixel capture unit 2031 creates and outputs digital original video information including still image information “320 pixels wide and 240 pixels high”.

[0464] The original video information is input to the video encoding unit 2035, and is first subjected to an encoding process by the video encoding unit main body 2036 to generate the internal encoded video information. Next, the resolution correction information adding unit 2037 adds the encoded video information created by the video encoding unit main unit 2036 to the encoded video information.
Information indicating "width 320 pixels and height 240 pixels" is added, and coded video information to be output from the video / audio coding device is created and output.

Here, as in the description of the eighteenth embodiment, it is assumed that tasks other than the video encoding and the audio encoding have been executed on the operating system of the general-purpose computer, and other tasks are executed by the CPU. Time is allocated, control is transferred to "other processes", and CPU
Time is consumed. As described in the eighteenth embodiment,
Capture of video and audio by video camera 2001, audio capture unit 2002, and video capture unit 20
The processing by 31 is performed substantially in parallel with the “other process”, and the sound is stored in the sound buffering unit 2003 up to 90%.

Thereafter, when the audio encoding process is performed, the audio encoding unit 2005
From 003, a certain amount of original audio information is read out first from the amount stored earlier (in the past), and the corresponding original audio information is deleted from the audio buffering unit 2003, and the original audio buffer amount 2
004 is updated. Further, the audio encoding unit 2005
Encode the original audio information. In the twentieth embodiment, it is assumed that the fixed amount to be read and deleted is 30%, and the audio encoding unit 2005 reads and encodes 30% of the original audio information stored up to 90% as described above. And
When the encoding is completed, the CPU time is released.

When the processing shifts to video encoding again, the encoding load evaluator 2032 determines that the original audio buffer amount 20
Check 04. Immediately after the reading of 30%, 60% of the original audio information is still stored. Therefore,
Since the value is greater than 50% of the reference value, the evaluation information is 0%, and “1” of the encoding load reference information 2010
Does not change even after the multiplication by
3 is 0%. Therefore, the video resolution information 2034 is
"160 pixels in width and 120 pixels in height". The video resolution information 2034 is output to the video capture unit 2031 and the video encoding unit 2035 as before.

[0468] The video capture unit 2031 receives analog video information and outputs it as digital original video information.
At this time, since the video resolution information 2034 is “width 160 pixels and height 120 pixels”, the video capture unit 2031 converts the digital original video information including still image information “width 160 pixels and height 120 pixels”. Output.

[0469] The original video information is input to the video encoding unit 2035, first subjected to encoding processing in the video encoding unit main unit 2036, and output as internal encoded video information. In the previous processing, the resolution is "320 pixels wide and 2 high.
Compared with "40 pixels", in the present processing, the resolution of the image is "160 pixels in width and 120 pixels in height", so that the information amount represented by the number of pixels is 1/4. Therefore, this encoding process is completed in one-fourth the time of the previous process. The resolution correction information adding unit 2037 adds information indicating “width 160 pixels, height 120 pixels” to the internal coded video information output from the video coding unit main unit 2036, and Is output as encoded video information which is the output of the device.

In FIG. 48, while video and audio are being captured by the video camera 2001, the video encoding and audio encoding processes are executed as described above, so that the video associated with the capture is processed. Voice coding is performed. And after the end of the video and audio capture,
The encoding also ends.

As described above, according to the video / audio coding apparatus of the twentieth embodiment, audio buffering section 2003
, Video capture unit 2031, encoding load evaluator 2
032, a video coding unit 2035 including a video coding unit main body 2036 and a resolution correction information adding unit 2037.
Is provided, the encoding load evaluator 2032 checks the amount of the unprocessed audio information stored in the audio buffering unit 2003, and according to the amount, the video resolution information 203
4 to control the resolution of the input video information, thereby controlling the amount of video information.
Therefore, the CPU time consumed for the video encoding process can be controlled, and the effect of the shortage of computer resources consumed by other applications and the video encoding unit itself is not easily recognized as a problem. It is possible to avoid a situation in which the audio is interrupted due to the video encoding, with only a reduction in the actual resolution.

In the twentieth embodiment, the video resolution information is changed on condition that the original audio buffer amount exceeds a certain value. However, other evaluation methods can be used. For example, the original audio buffer amount can be multiplied by a certain coefficient, and the resolution can always be set according to the buffer amount, and the effect of preventing interruption of the audio can be obtained similarly.

Embodiment 21 FIG. Embodiment 21 of the present invention
The video and audio encoding device according to the present invention prevents interruption of audio even when a load increases in software processing in a general-purpose computer or the like, as in the eighteenth embodiment, and uses the processing amount of audio data as an index. The encoding process of the video information is stopped.

FIG. 49 is a diagram showing a schematic configuration of a video / audio coding apparatus according to Embodiment 21 of the present invention. As shown in the figure, the video encoding device according to the twenty-first embodiment includes a video camera 2101, an audio capture unit 2102, an audio buffering unit 2103, an audio encoding unit 2142, a video capture unit 2106, a video encoding unit 2207, a codec. Load evaluator 2144 and system timer 2141
It is composed of Also, the encoded audio information and the encoded video information are output as the device output, as in the eighteenth embodiment.

[0475] In the figure, the audio encoding unit 2142
As in the first embodiment, the audio buffering unit 2103
Out of the original voices stored in the original voice, the earliest (previous) stored original voice is extracted, the extracted original voice is deleted from the audio buffering unit 2103, and the original voice is compression-coded and output as coded voice information. I do. In addition, the audio encoding unit 2142 according to the twenty-first embodiment holds and updates the processed audio information amount 2143 which is the total sum of the original audio extracted so far. The coding load evaluation unit 2144 obtains coding load evaluation information 2145 used for control of video coding by calculation in a method described later, and, in accordance with the obtained coding load evaluation information,
Indicate whether to execute the encoding of the original video information. The system timer 2141 measures the elapsed time of the encoding.

The video camera 2101, audio capture unit 2102, audio buffering unit 2103, video capture unit 2106, and video encoding unit 2107 are the same as those of the eighteenth embodiment.
And 1807, and the description is omitted.

In the calculation of the coding load evaluation information 1922 by the coding load evaluation unit 1821, first, the elapsed time of coding obtained from the system timer 2141 and
The input amount of the original voice is calculated using the input amount per time of the original voice information which is apparent in advance. Then, a predicted audio buffer amount is obtained as a difference between the original audio input amount obtained by the calculation and the processed audio information amount 2143 held by the audio encoding unit 2142. Next, using the obtained predicted audio buffer amount as the evaluation information, the coding load evaluation information 2145 is obtained by multiplication with the coding load reference information 2110 as in the eighteenth embodiment. Also in the twenty-first embodiment, the coding load reference information 2110 has a fixed value of “1”, and the predicted audio buffer amount becomes the coding load reference information 2110.
Then, the coding load evaluation unit 2144 uses the value of the coding load evaluation information 2145 to output the original video information to the video coding unit 2107 and execute the coding if the value does not exceed a certain amount. If it exceeds a certain amount, the encoding is not executed by discarding the original video information. Therefore, in the twenty-first embodiment, the encoding load evaluator compares the predicted buffer amount with the fixed amount, and the audio buffering unit 210
3 is set to 50% of the maximum buffer amount. Also,
The input amount of the original audio information per time is set to an amount such that the buffer of the audio buffering unit 2103 becomes maximum in 10 seconds.

The twenty-first embodiment thus constituted
The outline of the operation of the video / audio coding apparatus according to the above is as follows. That is, in the video / audio coding apparatus,
The audio encoding unit 2142 includes the audio buffering unit 210
3, the earliest (past) original sound is extracted from the original sound, the extracted original audio is deleted from the audio buffering unit 2103, and the processing is a sum of the original audios extracted so far. The updated audio information amount 2143 is updated, and the original audio is compression-encoded and output as encoded audio information. Then, the encoding load evaluator 2144 calculates the original audio input amount based on the elapsed time of the encoding obtained from the system timer 2141 and the input amount per hour of the original audio information that is apparent in advance, and the calculated original audio amount is calculated. A predicted audio buffer amount, which is a difference between the audio input amount and the processed audio information amount 2143, is obtained, and coding load evaluation information 2145 is obtained using the obtained predicted audio buffer amount. Then, video encoding is controlled according to the value of the encoding load evaluation information. The handling of voice is the same as in the eighteenth embodiment.

[0479] The operation of an example of encoding processing for a video and audio by the video and audio encoding apparatus according to the twenty-first embodiment will be described below. Here, Embodiment 1
8, the video / audio coding process includes video coding (processing of the coding load evaluator 2144 and the video coding unit 2107) and voice coding (the voice coding unit 2142) according to the control of the operating system in the general-purpose computer. Processing)
When each task to which CPU time is allocated executes a series of processing and releases computer resources (CPU time), the operating system gives the other tasks the CPU time. The assignment is controlled.

First, as in the eighteenth embodiment, the video camera 2101 fetches video / audio information, and separates and outputs analog audio information and analog video information. And
The audio capture unit 2102 receives the analog audio information output from the video camera 2101 and outputs it as digital original audio information. Audio buffering unit 2103
Accumulates the original audio information and updates the original audio buffer amount 2104 according to the accumulated amount. Meanwhile, the video capture unit 21
Reference numeral 06 inputs analog video information output from the video camera 2101 and outputs it as digital original video information.

[0481] The encoding load evaluator 2144 temporarily receives the original video information output from the video capture unit 2144, and confirms the predicted audio buffer amount at that time. At this time, the elapsed time obtained by referring to the system timer 2141 is 1 second, and the processed audio information amount 214
Since 3 is still “0”, the predicted audio buffer amount is 10
%, Which is less than a predetermined reference value of 50%, and the coding load reference information 2110 has a value “1” that does not need to be considered in the multiplication process. Becomes Therefore, the encoding load evaluating unit 2144 converts the original video information into the video encoding unit 21.
07, the video coding unit 2107 performs video coding processing on the original video information, and releases the CPU time when the coding processing is completed.

Here, as in the description of the eighteenth embodiment, it is assumed that tasks other than the video encoding and the audio encoding have been executed on the operating system of the general-purpose computer, and the other tasks are executed by the CPU. Time is allocated, control is transferred to "other processes", and CPU
Time is consumed. As described in the eighteenth embodiment,
Capture of video and audio by video camera 2101, audio capture unit 2102, and video capture unit 21
The processing by 06 is performed substantially in parallel with the “other process”, and the sound is stored in the sound buffering unit 2103 up to 90%.

Thereafter, when the audio encoding process is performed, the audio encoding unit 2142
A certain amount of original audio information is read out from the audio buffer 103 from the audio buffering unit 2103, and the original audio information is deleted from the audio buffering unit 2103.
Update 104. Further, the audio encoding unit 2142
Encode the original audio information. In the twenty-first embodiment, the fixed amount to be read and deleted is 30%, and the audio encoding unit 2105 reads and encodes 30% of the original audio information stored up to 90% as described above. And
At the end of the encoding, the amount of 30% of the processed audio information amount 2143 held by itself is updated by adding
Release U time.

When the processing shifts to video encoding again, the encoding load evaluator 2144 first refers to the system timer 2141 to check the elapsed time at that time. The elapsed time is 9 seconds because the process has been moved to the “other process”. Next, referring to the processed audio information amount 2143 held by the audio encoding unit 2142, it was 30%. For this reason, the predicted audio buffer amount is 60%, which is the reference value of 50%.
, The evaluation information is 0%, and the encoding load evaluation information obtained by multiplying the encoding load reference information 2110 by “1” is also 0%. Therefore, the coding load evaluator 2
In step 144, the original video information at that time is discarded, and the CPU time is immediately released so that the audio encoding is executed.

[0485] In Fig. 49, while video and audio are being captured by the video camera 2101, the video encoding and audio encoding processes are executed as described above. Voice coding is performed. And after the end of the video and audio capture,
The encoding also ends.

As described above, according to the video and audio coding apparatus of the twenty-first embodiment, the system timer 2141
Speech encoding unit 2 holding processed speech information amount 2143
142 and the encoded voice load evaluator 2144, the encoding load evaluator 2144
141, and the elapsed time obtained by referring to
Processed audio information amount 2143 obtained with reference to 142
Since the predicted buffer amount is calculated from the above, and the predicted buffer amount is used as a substitute for the original audio buffer amount 2104 to control the video encoding, control is performed using the original audio buffer amount according to the eighteenth embodiment. Similarly, it is possible to prevent interruption of audio due to lack of computer resources consumed by other applications or the video encoding unit itself.

In the twenty-first embodiment, unlike the eighteenth embodiment, even if the original audio buffer amount 2104 is not known, the currently stored audio data can be stored by referring to the information amount processed by the audio encoding unit 2142. Since it is possible to predict the original audio buffer amount 2104 that will be present, it is possible to easily cope with the use of an existing application in which the buffer unit is a black box.

In the twenty-first embodiment, in the configuration according to the eighteenth embodiment in which the video encoding process is stopped according to the situation, the video encoding using the predicted buffer amount based on the processed audio information amount as an index In this embodiment, the prediction buffer amount is used as an index, compared with Embodiment 19 in which the processing amount of inter-frame prediction coding is controlled and Embodiment 20 in which the resolution is changed. It is also possible to apply the method 21.
The same effect that control can be performed even when the user cannot know the information 04 is obtained.

Embodiment 22 FIG. Embodiment 20 of the present invention
The video and audio encoding device according to the present invention prevents interruption of audio even when a load increases in software processing in a general-purpose computer or the like, as in the eighteenth embodiment, and uses the audio encoding amount as an index. The encoding process of the video information is stopped.

FIG. 50 is a diagram showing a schematic configuration of a video / audio coding apparatus according to Embodiment 22 of the present invention. As shown in the figure, the video encoding device according to the twenty-second embodiment includes a video camera 2201, an audio capture unit 2202, an audio buffering unit 2203, an audio encoding unit 2205, a video capture unit 2206, a video encoding unit 2207, a code Load evaluator 2253 and system timer 2251
It is composed of Also, the encoded audio information and the encoded video information are output as the device output, as in the eighteenth embodiment.

In the figure, the coding load evaluator 2253
Is a method for obtaining a predicted audio buffer amount in the same manner as in the twenty-first embodiment and obtaining coding load evaluation information based on the calculated amount. A method for obtaining the predicted audio buffer amount is the encoding apparatus according to the twenty-first embodiment. Is different from In the twenty-second embodiment, encoding load estimating section 2253 outputs encoded audio amount 225 output from audio encoding section 2205.
2 is used, and the processed audio information amount 2254 obtained from the encoded audio amount 2252 is used instead of the processed audio amount 2143 in the twenty-first embodiment. As described above, the encoded audio information is output from the video / audio encoding device and is transmitted / recorded, and the amount thereof can be easily detected. Also in the coding load evaluator 2253 of the twenty-second embodiment, the point that the elapsed time is obtained from the system timer 2251, the point that the original sound input amount is obtained from the elapsed time and the amount of the original sound input per time, are fixed. The point that the encoding load reference information 2210 having a value of “1” is used is the same as in the twenty-first embodiment.

[0492] The video and audio coding apparatus according to the twenty-second embodiment differs from the video and audio coding apparatus except that the function of the coding load evaluator 2253 is different as described above, and that the sound coding unit 2205 does not hold the processed sound amount. Has a configuration similar to that of the video / audio coding apparatus according to Embodiment 21. Accordingly, regarding the video camera 2201, the audio capture unit 2202, the audio buffering unit 2203, the video capture unit 2206, and the video encoding unit 2207, 1801 to 1803 and 1806 of the eighteenth embodiment
And 1807, and the system timer 2251
Is the same as in the twenty-first embodiment, and a description thereof will be omitted.

In the twenty-second embodiment, as in the twenty-first embodiment, the fixed amount to be compared with the predicted audio buffer amount is 50% of the maximum buffer amount, and the input amount per hour of the original audio is 10 seconds. The amount is set so that the buffer of the audio buffering unit 2203 is maximized. Further, the compression rate of the audio encoding unit 2205 is set to 1/10.

The twenty-first embodiment thus constructed
The outline of the operation of the video / audio coding apparatus according to the above is as follows. That is, the encoding load evaluator 2253 of the video / audio encoding apparatus uses the elapsed time of the encoding obtained from the system timer 2251 and the input amount of the original audio information per time which is apparent in advance, to obtain the original audio input. Then, the amount of processed audio information 2254 is calculated from the amount of encoded audio information 2252, which is the total amount of encoded audio information output by the audio encoding unit 2205. A predicted audio buffer amount which is a difference from 2254 is obtained, and coding load evaluation information 2209 is obtained using the obtained predicted audio buffer amount. Then, the encoding load evaluator 2253 inputs the original video information to the video encoding unit 2207 if the predicted audio buffer amount is smaller than the predetermined amount, and discards the original video information and The processing of the encoding unit 2207 is interrupted, and the computer resources (CPU time) are given to the audio encoding unit 2205.

[0495] The operation of an example of encoding processing for a video and audio by the video and audio encoding apparatus according to the twenty-second embodiment will be described below. Here, Embodiment 1
8, the video / audio coding processing includes video coding (processing of the coding load evaluation unit 2253 and the video coding unit 2207) and voice coding (the voice coding unit 2205) according to the control of the operating system in the general-purpose computer. Processing)
When each task to which CPU time is allocated executes a series of processing and releases computer resources (CPU time), the operating system gives the other tasks the CPU time. The assignment is controlled.

First, as in the eighteenth embodiment, the video camera 2201 takes in video / audio information, and outputs it separately into analog audio information and analog video information. And
The audio capture unit 2202 receives the analog audio information output from the video camera 2201 and outputs it as digital original audio information. Audio buffering unit 2203
Stores the original audio information, and updates the original audio buffer amount 2204 according to the accumulated amount. Meanwhile, the video capture unit 22
Reference numeral 06 inputs analog video information output from the video camera 2201 and outputs it as digital original video information.

[0497] The encoding load evaluator 2244 once receives the original video information output from the video capture unit 2144, and confirms the predicted audio buffer amount at that time. At this time, the elapsed time obtained by referring to the system timer 2251 is 1 second, and the encoded audio volume 2252 is still “0”, so the predicted audio buffer volume is 10% and the predetermined reference value Is less than 50%, and the coding load reference information 2110 has a value “1” that does not need to be considered in the multiplication process, so the coding load evaluation information 2245 becomes 100%. Therefore, the encoding load evaluation unit 2253 converts the original video information into the video encoding unit 2207.
, The video encoding unit 2207 performs video encoding processing on the original video information, and releases the CPU time when the encoding processing ends.

Here, as in the description of the eighteenth embodiment, it is assumed that tasks other than video encoding and audio encoding have been executed on the operating system of the general-purpose computer, and the other tasks are executed by the CPU. Time is allocated, control is transferred to "other processes", and CPU
Time is consumed. As described in the eighteenth embodiment,
Capture of video and audio by video camera 2201, audio capture unit 2202, and video capture unit 22
The processing by 06 is performed substantially in parallel with the “other process”, and the sound is stored in the sound buffering unit 2203 up to 90%.

[0499] Thereafter, when the audio encoding process is performed, the audio encoding unit 2205 outputs
A certain amount of original audio information is read out from the audio buffer 203, and the corresponding original audio information is deleted from the audio buffering unit 2203.
Update 204. Further, the audio encoding unit 2205
Encode the original audio information. In the twenty-second embodiment, the fixed amount to be read and deleted is 30%, and the audio encoding unit 2205 reads and encodes 30% of the original audio information stored up to 90% as described above. And
At the end of the encoding, the CPU time is released.

When the processing shifts to video encoding again, the encoding load evaluator 2244 first refers to the system timer 2241 to check the elapsed time at that time. The elapsed time is 9 seconds because the process has been moved to the “other process”. Next, since the encoded audio amount output from the audio encoding unit 2205 is 3% of the buffer amount of the audio buffering unit 103 and the compression ratio is 1/10, the processed audio information amount 2254 is , 30%. For this reason,
The predicted audio buffer amount is 60%, which is the reference value of 50.
%, The evaluation information is 0%, and the coding load evaluation information obtained by multiplying the coding load reference information 2110 by “1” is also 0%. Thus, the encoding load evaluator 2144 discards the original video information at that time, immediately releases the CPU time, and performs audio encoding.

In FIG. 50, while the video and audio are being captured by the video camera 2201, the video encoding and audio encoding processes are executed as described above, whereby the video associated with the capture is processed. Voice coding is performed. And after the end of the video and audio capture,
The encoding also ends.

As described above, according to the video and audio coding apparatus of the twenty-second embodiment, the system timer 2251
From the encoded audio volume 2252 to the processed audio information volume 2254
And a coded voice load evaluator 2253 that obtains the
251 and the coded voice amount 2
Processed audio information amount 2254 obtained by referring to 252
Since the predicted buffer amount is calculated from the above and the predicted buffer amount is used as a substitute for the original audio buffer amount 2204 to control the video encoding, control is performed using the original audio buffer amount according to the eighteenth embodiment. Similarly, it is possible to prevent interruption of audio due to lack of computer resources consumed by other applications or the video encoding unit itself.

[0503] Further, unlike Embodiment 18 and Embodiment 21, Embodiment 22 differs from Embodiment 18 and Embodiment 21 in that even if the original audio buffer amount 2204 and the processing amount in the audio encoding unit 2253 are not known, audio encoding is not performed. By referring to the amount of information processed and output by the unit 2253, the original audio buffer amount 2204 that will be currently stored can be predicted. It is also possible to easily cope with using an existing application whose black box is also a black box.

[0504] In the twenty-second embodiment, in the eighteenth embodiment in which the video encoding process is stopped according to the situation, the control is performed using the predicted buffer amount based on the encoded audio amount as an index. It is also possible to apply the present method using the prediction buffer amount as an index to the nineteenth embodiment for controlling the processing amount of inter-frame prediction encoding and the twenty-second embodiment for changing the resolution. The effect is obtained.

Now, the first embodiment described so far.
Eighth to twenty-second embodiments are based on the premise that encoding is possible on average, and when computer resources are temporarily reduced due to the influence of an instantaneous or short-term load increase. In addition, this is for preventing interruption of audio. It is possible to apply the video / audio encoding apparatus according to these embodiments to a case where the apparatus is basically implemented by software that operates on a computer system having a low computational function. In such a case, various conditions and It may not be suitable for all codings in the object.

FIG. 51 is a diagram showing the transition of the audio buffer amount when the video / audio coding apparatus according to the eighteenth to twenty-second embodiments is realized on a computer system having basically low computational capability. . As shown in the figure, since the load of video encoding is too large as a whole, unprocessed audio information increases during video encoding. As a result, at the stage where the video encoding is completed, control for giving priority to the audio encoding is performed, and the audio encoding process is suddenly executed with priority, and the video encoding process is stopped during that time. When the processing returns to video encoding again when the amount of unprocessed audio information decreases, the amount of unprocessed audio information increases. When such a process is repeated, the encoded video information rapidly changes from high-quality information to low-quality information, and then returns to high-quality information. In that case, it becomes difficult to appreciate.

In order to solve such a problem when performing video coding according to Embodiments 18 to 22, a video coding apparatus according to Embodiments 23 and 24 of the present invention, which will be described below, has a basic configuration. It is realized on a computer having a poor ability in terms of performance, and can also suppress the above-described large fluctuation in the image quality of the video while also preventing interruption of the sound.

Embodiment 23 FIG. Embodiment 23 of the present invention
The video / audio coding apparatus according to the present invention is adapted to cope with the case where the video / audio coding processing is performed by software processing in a general-purpose computer or the like having low performance by setting the coding load reference information.

FIG. 52 is a diagram showing a schematic configuration of a video / audio coding apparatus according to Embodiment 23 of the present invention. As shown in the figure, the video encoding apparatus according to Embodiment 23 includes a video camera 2301, an audio capture unit 2302, an audio buffering unit 2303, an audio encoding unit 2305, a video capture unit 2306, a video encoding unit 2307, a codec. It comprises a coded load evaluation unit 2308, a system timer 2361, and a coded load reference setting unit 2362.
As in the eighteenth embodiment, encoded audio information and encoded video information are output as device outputs.

[0510] In the figure, a system timer 2361 measures the elapsed time. Encoding load reference setting unit 2362
Investigates the variation of the original audio buffer amount 2304 per unit time, and determines the encoding load reference information 23 according to the degree of the variation.
Set 63. The coding load evaluation unit 2308 calculates the coding load evaluation information 2309 using the coding load reference information set by the coding load reference setting unit 2382 instead of the coding load reference information having a fixed value. Get by. The video / audio coding apparatus according to Embodiment 23 is different from the video coding apparatus according to Embodiment 18 in that a system timer 2361 and a coding load reference setting unit 2362 are provided.
And a video camera 2301, an audio capture unit 2302, and an audio buffering unit 230.
3, audio encoding unit 2305, video capture unit 230
6 and the video encoding unit 2307 are the same as 1801 to 1807 of the eighteenth embodiment, and a description thereof will be omitted.

[0511] The encoding load criterion setting unit 2362 executes a counting operation of setting "1 count" when the original audio buffer amount 2304 exceeds a certain value or becomes smaller than the certain value, and executes a counting operation per unit time. When the count exceeds 3 counts, the coding load reference information 2363 is reset.

[0512] The coding load reference information 2363 is information indicating how much video processing is performed when the audio buffer is empty. For example, if the value of "1" representing 100% is used, the audio buffer is If it is empty, process the video by 10
0% is performed, and if the value is “0.5” representing 50%, the video processing is performed 50% when the audio buffer is empty.
It indicates that.

[0513] In the twenty-third embodiment, the coding load reference information 2363 has an initial value of "1". If the count exceeds 3 counts in the operation, the value is set to “0.5” by resetting.

[0514] The coding load evaluator 2308 obtains the coding load information 2309 by using this value. Therefore, when the coding load reference information 2363 is "1", the value of the coding load information 2309 is set to "1". As in the case of the eighteenth mode, the value is 0% or 100%, but when the coding load reference information 2363 is “0.5”, the value is 0% or 50%. 50%
, The coding evaluation unit 2308 compares the original video information input at that time with the 100% video coding unit 230
7 instead of inputting it to the video encoding unit 23 by 50%.
07. Therefore, in this case, instead of processing a so-called full frame (30 fps), 15 fp
The processing of s will be performed.

The twenty-third embodiment thus constituted
The outline of the operation of the video / audio coding apparatus according to the above is as follows. That is, in the video / audio coding apparatus,
The encoding load criterion setting unit 2362 includes a system timer 23
Based on the timed output of 61, a change in the original audio buffer amount 2304 per unit time is investigated, and the coding load reference information 2363 is set according to the degree of the change. The encoding load evaluator 2308 determines the set encoding load reference information 2363.
Is used to calculate the coding load evaluation information 2309, and according to the calculated value of the coding load evaluation information 2309,
The original video information is input to the video encoding unit 2307, or the original video information is discarded, the processing of the video encoding unit 2307 is interrupted, and the computer resources (CPU time) are reduced to the audio encoding unit 2305.
To decide whether to surrender or not. Therefore, even when the encoding processing of the original video information is performed in the video encoding unit 2307, not all the original video information is processed, and the processing is performed at a rate according to the situation. .

[0516] The operation of an example of encoding processing for a video and audio by the video and audio encoding apparatus according to the twenty-third embodiment will be described below. Here, Embodiment 1
8, the video / audio coding processing includes video coding (processing of the coding load evaluation unit 2308 and the video coding unit 2307) and voice coding (the voice coding unit 2305) according to the control of the operating system in the general-purpose computer. Processing)
When each task to which CPU time is allocated executes a series of processing and releases computer resources (CPU time), the operating system gives the other tasks the CPU time. The assignment is controlled. Also, it is assumed that the basic capability of the general-purpose computer for realizing the video / audio coding apparatus is lower than that for realizing the coding apparatus according to the eighteenth embodiment.

First, as in the eighteenth embodiment, the video camera 2301 takes in video and audio information, and outputs it separately into analog audio information and analog video information. And
The audio capture unit 2302 receives the analog audio information output from the video camera 2301 and outputs it as digital original audio information. Audio buffering unit 2303
Stores the original audio information, and updates the original audio buffer amount 2304 according to the accumulated amount. Meanwhile, the video capture unit 23
Reference numeral 06 inputs analog video information output from the video camera 2301 and outputs it as digital original video information.

[0518] The encoding load evaluator 2308 receives the original video information once output from the video capture unit 2306, and checks the original audio buffer amount 2304 and the encoding load reference information 2363 at that time. At this point, the original audio buffer amount 2304 is 10%, which is lower than the predetermined reference value of 50%.
And the coding load reference information 2363 is “1”, which is the initial value.
0%. Therefore, the encoding load evaluator 2308 inputs all the frames of the original video information to the video encoder 2307, and the video encoder 2307 performs the video encoding process. To release.

As in the case of the eighteenth embodiment, the operations of the video camera 2301 and the capture board for realizing the audio capture unit 2302 and the video capture unit 2306 are performed substantially in parallel with the process of each task under CPU control. The original audio information can be stored in the audio buffering unit 2303 up to 90% because the CPU time is large in the video encoding process.

[0520] The audio encoding unit 2305 reads a fixed amount (original audio readout amount) of original audio information from the audio buffering unit 2303 from the previously (past) accumulated amount of original audio information, and converts the read original audio information into audio. Buffering unit 23
03, and the original audio buffer amount 2304 is updated. Further, the audio encoding unit 2305 encodes the original audio information. In the twenty-third embodiment, the original audio readout amount is set to 30% of the maximum buffer amount. As described above, 90% of the original audio information is stored in the audio buffering unit 2303. 30% is read, the original audio buffer amount 2304 is updated to 60%, encoding processing of the original audio information is performed, and the CPU time is released when the encoding is completed. Also, the encoding load reference setting unit 2362 that monitors the original audio buffer amount 2304
Since the original audio buffer amount 2304 is 60%, it is recognized that the original audio buffer amount 2304 exceeds the default value of 50%.

When the processing shifts to video encoding again, the encoding load evaluator 2308 determines that the original audio buffer amount 23
04 and the coded load reference information 2363 are confirmed. At this point, the original audio buffer amount 2304 is 60%, which is 50% or more of the predetermined reference value, so that the evaluation value becomes 0%, and the coding load reference information 2363 is added.
Is multiplied by “1”, the coding load evaluation information 2309 is 0.
%. Therefore, the encoding load evaluator 2308 discards all frames of the original video information at this time and immediately releases the CPU time. Therefore, in this video encoding process, the original audio buffer amount 2304 does not change.

[0522] The audio encoding unit 2305 reads a predetermined amount (original audio readout amount) of original audio information from the audio buffering unit 2303 from the earlier (past) amount of original audio information, and outputs the read original audio information. Buffering unit 23
03, and the original audio buffer amount 2304 is updated. Further, the audio encoding unit 2305 encodes the original audio information. The original audio reading amount is 30%, and the original audio information is stored in the audio buffering unit 2303 for 60%. Therefore, 30% of the original audio information is read out, and the original audio buffer amount 2304 is updated to 30%. The audio information is encoded, and the CPU time is released when the encoding is completed.

At this point, the encoding load criterion setting unit 2362 monitoring the original audio buffer amount 2304 has the default value of 50 since the original audio buffer amount 2304 is 30%.
%, And together with the fact that it has exceeded the predetermined value the previous time, it is recognized that the fluctuation of the original audio buffer amount 2304 is 1 count. That is, a count operation for one count is performed.

[0524] By repeating such a process, when the count operation of the coded voice reference setting unit 2362 reaches three counts due to the fluctuation of the original voice buffer amount 2304, the coding load reference setting unit 2362 sets the The load reference information 2363 is reset from the initial value “1” to “0.5” indicating 15 fps.

When the processing shifts to video encoding again, the encoding load evaluator 2308 determines that the original audio buffer amount 23
04 and the coded load reference information 2363 are confirmed. Here, since the original audio buffer amount 2304 is 30%, the evaluation value is 100%. However, the coding load reference information 2
Since 363 is “0.5”, the multiplication results in 50% of the coding load reference information 2363. Therefore, half of the frames of the original video information are discarded by discarding, and the remaining frames are input to the video encoding unit 2307, where the video encoding process is performed, and the CPU time is released when the processing is completed.

[0526] The audio encoding unit 2305 outputs a fixed amount (30 in the twenty-third embodiment) from the audio buffering unit 2303.
%) Of the original audio information is read out first from the previously (previously) accumulated original audio information, and the corresponding original audio information is deleted from the audio buffering unit 2303, and the original audio buffer amount 2304 is updated. Further, the audio encoding unit 2305 encodes the original audio information. In this process, the time required for the previous video encoding process is about half that required before the reset of the encoding load reference information.
03 stores 60% of voice. 30 of them
% Is read out and set to 30%, and when encoding is completed, C
Release PU time.

When the processing shifts to video encoding again, the encoding load evaluator 2308 determines that the original audio buffer amount 23
04 and the coded load reference information 2363 are confirmed. At this point, the original audio buffer amount 2304 is 30%, which is lower than the default value. However, since the encoding load reference information 2363 is 15 fps, of the frames of the original video information,
Half is discarded and discarded, and the remaining frames are input to the video encoding unit 2307, where the video encoding process is performed, and the CPU time is released when the process is completed.

In FIG. 52, while the video and audio are being captured from the video camera 2301, the video encoding and audio encoding processes are executed as described above, whereby the video associated with the capture is obtained. Voice coding is performed. Then, after the capturing of the video and audio ends, the encoding ends.

FIG. 53 is a diagram showing the operation of the video / audio coding apparatus according to the twenty-third embodiment over a long period of time in coding. As shown in FIG.
In the section, there is a large variation in the audio buffer amount as shown in FIG. 54, and as shown in the case where the eighteenth to twenty-second embodiments are executed on a computer with low performance, video encoding and audio priority are given. Is repeated. However, as described above, the encoding load reference information 2363 is reset during this time. In the B section, the reference of the video encoding load has been reduced by half. It is well balanced.

As described above, according to the video and audio coding apparatus of the twenty-third embodiment, the system timer 2361 and the coding load reference setting section 2362 are added to the video and audio coding apparatus of the eighteenth embodiment. The encoding load criterion setting unit 2362 determines that the original audio buffer amount 2304
In response to the fluctuation of the coding load reference information 2363, the coding load evaluation unit 2306 obtains the coding load evaluation information 2309 using the coding load reference information 2363, so that the Then, the rate of the encoding process of the original video information is changed. Therefore, in addition to being able to prevent audio from being interrupted due to load fluctuations, when encoding video and audio using software that operates on a computer system with basically low computational power, the optimal video load in that system is reduced. The setting is automatically performed, so that it is possible to avoid a situation in which high-quality video and low-quality video are repeated.

[0531] In the twenty-third embodiment, in the eighteenth embodiment in which the video encoding processing is stopped according to the situation, the control by changing the encoding load reference information based on the fluctuation of the original audio information storage amount is performed. However, it is also possible to apply the present method of changing the coding load reference information to the embodiment 19 for controlling the processing amount of the inter-frame prediction coding and the embodiment 20 for changing the resolution. It is possible, and the same effect as in the twenty-third embodiment can be obtained.

For example, when applied to the nineteenth embodiment, the coding load reference information is used to calculate the motion vector, and how much coding should be performed when the original audio buffer amount is “0”. If the fluctuation of the original audio buffer amount is large, even if the original audio buffer amount is “0”, the motion vector can be calculated by 50% of the set value. It is. Therefore, by such an application, also in the nineteenth embodiment,
It is possible to avoid a situation where high-quality video and low-quality video are repeated.

In the twenty-third embodiment, the actual coding is performed using the coding load reference information of the initial value, and suitable coding load reference information is set in accordance with the fluctuation of the original audio buffer amount. However, if the encoding load reference information obtained by setting is stored in a storage device such as a hard disk, the next time encoding is performed, the encoding can be performed with suitable encoding load reference information from the beginning. I can do it. That is, although a period occurs during the first time in which a suitable encoding cannot be performed as in the A section shown in FIG. 53, the next time, a well-balanced encoding as shown in the B section can be performed.

Embodiment 24 FIG. Embodiment 24 of the present invention
The video and audio encoding device according to the present invention is to cope with the case where the video and audio encoding processing is performed by software processing in a general-purpose computer or the like that is not high-performance, by setting the encoding load reference information, and as a result of the setting. Can be presented to the user.

FIG. 54 is a diagram showing a schematic configuration of a video / audio coding apparatus according to Embodiment 24 of the present invention. As shown in the figure, the video encoding apparatus according to Embodiment 24 includes a video camera 2401, an audio capture unit 2402, an audio buffering unit 2403, an audio encoding unit 2405, a video capture unit 2406, a video encoding unit 2407, a codec. Load evaluation section 2408, system timer 2461, coding load reference setting section 2462, coding load presentation section 241
1 and a standard video / audio output unit 2412 for load setting. As in the eighteenth embodiment, encoded audio information and encoded video information are output as device outputs. Further, the encoding load presentation unit 2411 performs output to a monitor.

In the figure, an encoding load presentation unit 2411
Presents the setting result of the coding load reference information 2463 to the user of the video / audio coding apparatus. The load setting standard video / audio output unit 2412 outputs standard video information and audio information in order to set a video load in accordance with the computing capability of the computer system. The video / audio encoding apparatus according to Embodiment 24 has a configuration in which an encoding load presentation unit 2411 and a standard video / audio output unit for load setting 2412 are added to the video encoding apparatus according to Embodiment 23, and a video camera 2401, audio capture unit 2
402, an audio buffering unit 2403, an audio encoding unit 2405, a video capturing unit 2406, and an image encoding unit 2407 are the same as those of the eighteenth embodiment.
807, and a system timer 2461
And the coding load criterion setting unit 2462 correspond to the second embodiment.
3, the description is omitted.

The twenty-fourth embodiment thus configured
The outline of the operation of the video / audio coding apparatus according to the above is as follows. That is, in the video / audio coding apparatus,
The encoding load criterion setting unit 2462 includes the system timer 24
Based on the clock output of 61, a change in the original audio buffer amount 2404 per unit time is investigated, and the coding load reference information 2463 is set according to the degree of the change. The encoding load evaluator 2408 determines the set encoding load reference information 2463.
Is used to calculate the encoding load evaluation information 2409, and according to the calculated value of the encoding load evaluation information 2409,
The original video information is input to the video encoding unit 2407, or the original video information is discarded, the processing of the video encoding unit 2407 is interrupted, and the computer resources (CPU time) are reduced by the audio encoding unit 2405.
To decide whether to surrender or not. Therefore, even when the encoding processing of the original video information is performed in the video encoding unit 2407, not all the original video information is processed, and the processing is performed at a rate according to the situation. .

In addition, the load setting standard video output unit 241
2 outputs standard video information and audio information based on the standard video / audio information 2413 in order to set the video load in accordance with the computing function of the computer system.
Also, the encoding load presentation unit 2411 presents the setting result of the encoding load reference information 2463 to the user of the video / audio encoding device.

[0539] The operation of an example of encoding processing for a video and audio by the video and audio encoding apparatus according to the twenty-fourth embodiment will be described below. Here, Embodiment 1
8, the video / audio encoding process includes video encoding (processing of the encoding load evaluator 2408 and the video encoder 2407) and audio encoding (the audio encoding unit 2405) according to the control of the operating system in the general-purpose computer. Processing)
When each task to which CPU time is allocated executes a series of processing and releases computer resources (CPU time), the operating system gives the other tasks the CPU time. The assignment is controlled. Also, it is assumed that the basic capability of the general-purpose computer for realizing the video / audio coding apparatus is lower than that for realizing the coding apparatus according to the eighteenth embodiment.

First, the load setting standard video / audio output unit 24
12 outputs standard video / audio information 2413 prior to actual coding of video information. Similarly to the video / audio information taken in the twenty-third embodiment, the coding apparatus according to the twenty-fourth embodiment codes the standard video / audio information 2413, and the coding load reference setting unit 2462 outputs the coding load reference information 2463. Set. Encoding load presentation unit 2411
Presents the contents of the coded load reference information 2463 to the user through the monitor, and obtains the consent. The load setting standard video output unit 2412 switches the video information and audio information to be output to the video information and audio information from the video camera 2401, and performs normal encoding as described in Embodiment 23. .

In Embodiment 24, standard video / audio information prepared in advance is used to set the coding load reference information. However, the present invention is limited to such a method. Not something. For example, it is possible for a user to use arbitrary video and audio information for evaluation in accordance with a video to be encoded. Also, for the standard video / audio information used in the twenty-fourth embodiment, the audio coding in the video / audio coding apparatus is performed with a constant coding regardless of the content of the audio information (such as voiced / silent). As long as the processing is performed, silent data can be used as the audio constituting the standard video / audio information.

As described above, according to the video / audio coding apparatus of the twenty-fourth embodiment, the video / audio coding apparatus of the twenty-third embodiment has a load setting standard video / audio output unit 2412 and an encoding load display unit. By adding 2411 to the configuration, prior to actual encoding processing, first, standard video / audio information is encoded, thereby setting encoding load reference information, and presenting it to the user. With this configuration, the user can perform the encoding process after being convinced that the quality of the video has deteriorated. That is, the encoding device using software as described in the twenty-fourth embodiment can operate on various computer systems. Therefore, it operates in various environments, from those with originally high computational capabilities to those with low computational capabilities.

As described above, one of the objects of the present invention is to perform audio coding without interruption without sacrificing the quality of video, regardless of the magnitude of the computing function. At 24, the user can be shown how much the video quality has been sacrificed. As a result, the user can recognize that the deterioration of the image quality is caused by the lack of the computational capability of the computer system, and take measures such as improving the operating frequency or increasing the main memory. be able to. Therefore, according to the twenty-fourth embodiment, in addition to the effects shown in the twenty-third embodiment, a special effect that a user can be notified of the situation of the computer system can be obtained.

[0544] The present invention is not limited to the above embodiments. For example, in each of the above embodiments, the audio encoding unit 2405 is replaced by the audio buffering unit 2
Although the original audio information stored in 403 is read out, the read original audio information is deleted from the audio buffering unit 2403, the original audio information is encoded, and the encoded audio information is output as encoded audio information. Voice encoding section 240
5 is read out of the original audio information stored in the audio buffering unit 2403, the read original audio information is encoded, and output as encoded audio information. You may be comprised so that it may be deleted. Further, after detecting that the audio encoding unit 2405 has read the original audio information stored in the audio buffering unit 2403, a deletion unit for deleting the read original audio information from the audio buffering unit 2403 is separately provided. Is also good. In addition, it is possible to make various design changes and modifications within the scope of the claims of the present invention.

[0545] Also, with regard to the video and audio coding method described in the eighteenth to twenty-fourth embodiments, a personal computer, a workstation, or the like uses a recording medium on which a video and audio coding program capable of executing the method is recorded.
This can be realized by executing the program.

Embodiment 25 FIG. Embodiment 25 of the present invention
Is a method for determining an encoding parameter according to a setting in the same manner as in the first embodiment. It is for determining other parameters.

In the first to fourth embodiments, the frame rate is specified, but in the twenty-fifth embodiment,
An encoding process is performed at a frame rate as high as possible without specifying a frame rate, thereby obtaining encoded data with high reproduction image quality. In the twenty-fifth embodiment, the resolution of the input image data is given, and the encoding process is performed in accordance with the given resolution.

FIG. 55 is a block diagram showing a configuration of a video encoding apparatus according to Embodiment 25 of the present invention. As shown in the figure, the video encoding apparatus according to Embodiment 25
Encoding means 3001 and encoding parameter determination means 30
02, and the encoding means 3001
CT processing means 3003, quantization means 3004, variable length coding means 3005, bit stream generation means 300
6. Inverse quantization means 3007, inverse DCT processing means 300
8 and the predicted image generation means 3009, and the coding parameter determination means 3002 include a motion vector (MV) detection range reference table 3010.

The encoding means 3001 inputs video data consisting of a series of still images obtained by digitizing the video as input image data, performs an encoding process in accordance with the set encoding parameters, and outputs encoded data. . Each still image data constituting the input image data is called a frame image. Further, the encoding parameter is provided from an encoding parameter determining unit 3002 described later.
A parameter indicating an encoding type and a parameter indicating a detection range of a motion vector are included. The parameter indicating the encoding type indicates intra-frame encoding processing or forward prediction encoding processing.
01 is intra-frame encoding,
Alternatively, forward prediction coding is performed. A motion vector used for forward prediction coding is detected within a range indicated by a parameter indicating a detection range of the motion vector.

The DCT unit 3 inside the encoding unit 3001
003, quantization means 3004, variable length coding means 300
5. The bit stream generation means 3006, the inverse quantization means 3007, and the inverse DCT means 3008 are the same as 103 to 108 in the first embodiment and will not be described.

The predictive image generating means 3009 receives the inverse DCT transform data output from the inverse DCT processing means 3008 and performs a motion vector detection process between the inverse DCT transform data and the input image data. A predicted image is generated and output as predicted image data. The detection of the motion vector is performed within the range indicated by the parameter indicating the detection range of the motion vector as described above. When the inter-frame encoding process using the predicted image is performed, the difference data between the predicted image data and the input image data is DC
When input to the T means 3003, the encoding means 3
In 001, forward prediction encoding is performed.

In the video coding apparatus according to the twenty-fifth embodiment, the coding parameter determining means 3002 determines the MV detection range reference table 3010 included in the MV detection range reference table from the resolution of the input image data and the specified coding pattern.
Is used to determine the MV detection range, and outputs the above-mentioned encoding parameter including a parameter indicating the determined MV detection range to the encoding means 3001.

In the video coding apparatus according to the twenty-fifth embodiment, as in the first embodiment, a personal computer (PC) executes a video coding program under the control of a processing control unit (CPU). In the execution of the encoding process, the following two conditions are satisfied in addition to the five conditions shown in the first embodiment.

(6) When forward encoding processing is performed,
If the detection range of the motion vector is “small”, the processing time is six times that in the case of performing the intra-frame encoding process.

(7) When the forward encoding process is performed,
If the detection range of the motion vector is “large”, the processing time is four times that of the case where the detection range is “small”.

Here, the operating frequency of the CPU mounted on this apparatus is 100 MHz, the frame rate specified at the start of coding is 24 frames / sec, and the coding pattern as a combination of coding types is 2 frames. It is assumed that pattern 2 “IP” that repeats “I” and “P” every time is used. Here, the intra-frame encoding is represented by “I” and the forward prediction encoding is represented by “P”.

[0556] The operation of the video encoding apparatus according to the twenty-fifth embodiment configured as described above based on the above settings will be described below. First, a video to be encoded is digitized and input to the encoding unit 3001 of the encoding device as a series of frame images. FIG. 56 is a flowchart showing the operation of the encoding means 3001.
The operation of the encoding means 3001 will be described below with reference to FIG. Note that the coding parameter determining means 3002
Is to instruct the encoding means 3001 to perform intra-frame encoding for the first frame image at the start of encoding.

At step D01, the encoding parameters inputted from the encoding parameter determining means 3002 are judged. If the conversion has been instructed, the processing after step D07 is executed.

In the case where step D02 and subsequent steps are executed,
Steps D02 to D05 are executed in the same manner as steps A02 to A05 in the first embodiment. In step D06, it is determined whether or not the encoding has been completed. If it is determined that the encoding has been completed, the process ends. On the other hand, if the encoding is not completed, the process returns to step D01, and the steps after step D01 are executed.

On the other hand, when step D07 and subsequent steps are executed according to the judgment in step D01, the following is performed. First, in step D07, the inverse quantization means 3007
The inverse quantization unit 3004 inversely quantizes the quantized data already output for the immediately preceding frame image by the quantization unit 3004, and outputs inversely quantized data. Next, in step D08, the inverse D
CT processing means 3008 calculates D
The inverse processing of the two-dimensional discrete cosine transform is performed for each block of 8 × 8 pixels divided by the CT processing unit 3003.
Performs a dimensional inverse discrete cosine transform and outputs inverse DCT transform data. In step D09, the prediction image generation unit 3009 generates a prediction image (uncompensated) based on the inverse DCT transform data, and generates a prediction image (uncompensated) based on the generated image and the input image data in a range specified by the encoding parameter. Then, a motion vector is detected, and the motion vector is used to generate and output a motion-compensated predicted image.

In step D10, DCT processing means 3003
Represents the input frame image and the predicted image generation means 300
9 is divided into blocks of 8 pixels × 8 pixels based on the specified resolutions, and for each of the divided blocks, the data of the predicted image is subtracted from the data of the input frame image. Obtain difference data. Then, two-dimensional discrete cosine transform is performed on the difference data for each divided block, and DCT transform data is output. Steps D11 to D14 after the DCT conversion data is output are the same as steps D03 to D14 described above.
It is executed in the same way as 06.

As described above, the encoding means 3001 determines whether or not steps D02 to D06 or steps D07 to D07 are determined for each input frame image in step D01.
Fourteen processes will be performed. Step D02-
D06 is intra-frame encoding, and steps D07 to D07 are performed.
14 performs forward coding processing based on a predicted image using the coding result for the immediately preceding frame image, and this switching is performed in the determination of step D01.
This is performed according to the input encoding parameters.

(Table 15) shows the encoding parameter determining means 3
002 is a table showing an MV detection range reference table 3010 included therein. FIG. 57 is a flowchart showing the operation of the encoding parameter determining means 3002. In the following, an encoding parameter is determined and the encoding means 3001 is determined.
The operation of the encoding parameter determining means 3002 for outputting to the.

[0564]

[Table 15]

The MV detection range reference table 3010 shown in (Table 15) is created in advance prior to encoding. The table can be created, for example, based on empirical knowledge or using results of experimental coding, simulation, or the like, in consideration of the conditions described below. The “input” column in (Table 1) indicates the resolution of the input image data and the designated parameter, and the “output” column indicates the parameter determined in response to the input. As shown in the table, in the twenty-fifth embodiment, in accordance with the resolution of the input image data and the
The V detection range is determined. The encoding pattern is fixed to “IP”, and the pattern “I”
"P" means that intra-frame encoding (I) and forward prediction encoding (P) are repeated every two frames. As the resolution of the input image, “160 × 120” and “80”
× 64 ”.

[0566] Creation of the reference table is performed in consideration of the following conditions. First, the processing amount increases when the motion vector detection range increases, and second, the processing amount increases when the resolution of the input image is high as compared with the case where the resolution is low.

In consideration of these conditions, the MV detection range reference table 3010 is created so that detection is performed in a range as large as possible with respect to the resolution of the specified input image, and coded data with a high compression ratio can be obtained. Is what is done.

First, in step E01 of the flow in FIG. 57, the coding parameter determining means 3002 refers to the MV detection range reference table 3010 from the designated input image resolution and coding pattern (IP). , A motion vector detection range in predictive coding is determined.

Next, in step E02, the coding parameter determination means 3002 instructs the coding means 3001 on the MV detection range determined in step E01 and performs processing so as to realize the specified coding pattern. Indicate the encoding type (I or P) to be used for the target frame image.

[0570] Thereafter, in step E03, it is determined whether or not the encoding has been completed. If it is determined that the encoding has been completed, the process ends. On the other hand, if the processing has not been completed, the process returns to step E02, whereby the output of the coding parameters to the coding means 3001 is repeated.

[0571] Encoding is performed by the above-described operations of the encoding means 3001 and the encoding parameter determining means 3002. It is a table | surface which shows the result of having performed conversion.

[0572]

[Table 16]

(Table 16) shows that, for the specified conditions,
MV determined in encoding apparatus of Embodiment 25
It shows a detection range (determined parameters) and a frame rate (encoding result) obtained as a result of an encoding process using those parameters. For the numerical values of the encoding results shown in (Table 16), the encoding pattern “I
P ”, when the resolution is set to 160 × 120,
The frame rate in other cases is calculated based on the fact that processing can be performed at 7.4 frames / sec. The frame rate when the encoding pattern is IP, the resolution is 80 × 64, and the detection range of the motion vector is “large” requires that the processing when the detection range is large takes four times as long as the processing when the detection range is small. When the resolution is about 1/4, processing can be performed in about 1/4 time, so that it can be calculated as about 27.4 frames / sec.

For comparison, Table 17 shows the operation results when encoding is performed using a video encoding device according to the conventional technique.

[0575]

[Table 17]

In Table 17, the same calculation as in Table 16 is performed. In the case of the encoding pattern “IP”, when the resolution is 160 × 120 and the detection range is “small”, 27.4. Other frame rates have been calculated based on the ability to process at frames per second.

[0577] In the video coding apparatus according to the conventional technique, without considering the frame rate obtained as a coding result,
This is to determine the motion vector detection range. Therefore, it is often difficult to set the frame rate obtained as a result of the encoding process to be sufficiently high. On the other hand, in the video coding apparatus according to the twenty-fifth embodiment, in consideration of the frame rate as a coding result, the video coding apparatus according to the specified coding type (pattern) and the resolution of the input image have As shown in a comparison between Tables 16 and 17, by determining the detection range of the motion vector, it is possible to realize a frame rate as high as possible and obtain encoded data with a higher compression rate. It can be seen that the encoding is executed with the detection range set.

[0578] As described above, according to the video coding apparatus of the twenty-fifth embodiment, the coding apparatus 3001 and the coding parameter determining means 3002 including the motion vector detection range reference table 3010 are provided. The encoding parameter determination unit 3002 determines a detection range of a motion vector according to the designated encoding type and the resolution of the input image, and outputs an encoding parameter to the encoding unit 3001. Since the encoding unit 3001 performs the encoding process according to the encoding parameter, it is possible to perform the encoding that can obtain the encoded data with a higher compression ratio while realizing the required condition.

In the video coding apparatus according to the twenty-fifth embodiment, the detection range of the motion vector is determined according to the specified coding pattern and the resolution of the input image. By using, it is also possible to determine the presence or absence of filtering corresponding to the resolution of the input image, it is possible to perform processing to obtain a higher quality encoding result under the set conditions Become.

[0580] As in the first embodiment, the video coding method shown in the twenty-fifth embodiment uses a recording medium storing a video / audio coding program capable of executing the method and uses a personal computer or a computer. This can be realized by executing the program on a workstation or the like.

Also, in any of Embodiments 1 to 25, the medium for recording the encoding program includes:
Floppy disk, CD-ROM, magneto-optical disk,
Any computer that can record the encoding program, such as a phase change optical disk, and can be read and executed by a general-purpose computer such as a personal computer can be used.
Further, it is also possible to adopt an operation mode in which the program recorded in a storage device managed by another computer connected to the network is read out via the network, and the read out computer executes the program.

[0582]

According to the video encoding method of the present invention, one or more of the above-mentioned still image information is converted to the original video information composed of a plurality of still image information in which the video is digitized. A video encoding step of encoding according to the encoding parameters to be encoded, a resolution of the original video information, a frame rate required when reproducing encoded data obtained by encoding, and an encoding for executing the video encoding step. One or more of the above based on processing performance indicating the processing capability of the device, or one or more of one or a plurality of coding parameters affecting the processing amount of the coding process in the video coding step. And a coding parameter determination step of determining a coding parameter, so that a high resolution or a high compression ratio is obtained at a given frame rate. By setting the encoding parameters obtained in Goka result, it is possible to utilize the device resources, obtain good coding results.

According to the video encoding method of claim 2, in the video encoding method of claim 1,
Since the processing capability of the encoding device that executes the video encoding step is determined and the processing capability determination step of outputting the determination result is further performed, it is possible to obtain a good encoding result according to the processing capability. Become.

[0584] According to the video encoding method of claim 3, in the method of claim 1 or 2, the encoding parameter is a resolution in an encoding process performed on the original video information, an intra-frame encoding. Or, by including one or more of the encoding type indicating predictive encoding, or a detection range when detecting the motion vector used for the predictive encoding, to determine these parameters according to the setting, A good encoding result can be obtained by utilizing the device resources.

According to the video encoding method of the fourth aspect, in the method of the third aspect, in the processing capability determining step, the determination is performed based on the type of the control device of the video encoding method. By judging the hardware capability of the encoding device and setting the encoding parameter according to the capability, it is possible to obtain a good encoding result by utilizing the device resources.

According to the video encoding method of claim 5, in the method of claim 3, the above-mentioned determination is made based on the time required for the encoding process in the above-mentioned encoding step. By judging the encoding processing capability of the encoding device, indicating the required time, and setting an encoding parameter according to the capability, it is possible to obtain a good encoding result by utilizing the device resources. Becomes

According to the video encoding method of claim 6, in the method of claim 3, in the processing capacity determination step, the input original video information is temporarily stored, and the original video information is stored in the storage. A video buffering step of sequentially saving a series of still image information constituting the video information, and sequentially discarding the still image information read out in the encoding step and subjected to the encoding process; Performing a frame rate control step of controlling the storage of the series of still image information in the video buffering step to be performed at a constant frame rate determined based on the given frame rate. Since the above determination is made based on the storage amount of the original video information temporarily stored in the step, Indicated amount, determines the encoding capability of the encoder, by setting encoding parameters in accordance with their capabilities, it is possible to obtain good coding result by utilizing the device resource.

According to the audio encoding method of the present invention, in the audio encoding method for encoding audio by the band division encoding method, the set frequency fs, which is a numerical value used for the encoding process, is used. And a storage step of storing a conversion constant n; a voice input step of inputting a voice to be encoded; and a sampling frequency determined based on the stored set frequency fs. The input voice sampling step to be created and the number of sampled voice data obtained when the set frequency fs is used as the sampling frequency is m
And the number determined based on the conversion constant n is m ′
An audio data conversion step of outputting converted audio data composed of m audio data, including m ′ sampled audio data, and a band division of dividing the converted audio data to obtain M band signals A coding step for allocating coding bits only to a band signal having a frequency equal to or lower than the limit frequency among the band signals, using a frequency fs / 2n obtained from the stored set frequency fs and the conversion constant n as a limit frequency. And a quantization step of performing quantization based on the allocated coded bits, a coding step of outputting the quantized data as coded data, and coded data for recording the output coded data. Perform the recording step and
By setting the conversion constant n, the processing load is reduced, and the real-time encoding process accompanying the voice capture is performed.
It is possible to obtain an encoding result of sound quality according to the performance of the encoding device.

[0589] According to the speech encoding method of claim 8, in the method of claim 7, in the input speech sampling step, the input speech is sampled by using the stored set frequency fs as a sampling frequency. In the audio data conversion step, sampling audio data is extracted at intervals of (n-1) from the m sampling audio data, and two adjacent audio data are extracted. Between the extracted sampled audio data, (n-1)
Is converted into m pieces of converted voice data by inserting the voice data, thereby obtaining the converted voice data that can reduce the processing load, and performing the real-time coding process accompanying the voice capture and performing the coding. It is possible to obtain an encoding result of sound quality according to the performance of the device.

[0590] According to a ninth aspect of the present invention, in the method of the ninth aspect, in the audio data converting step, converted audio data in which the extracted sampled audio data is continuous by n pieces each is created. This makes it possible to easily obtain converted audio data that can reduce the processing load.

[0591] According to the speech encoding method of claim 10, in the method of claim 7, in the input speech sampling step, the frequency fs / n obtained from the stored set frequency fs and the conversion constant n is calculated. As the sampling frequency, m / n pieces of sampled audio data are created by sampling the input audio. In the audio data conversion step, two adjacent sampled audio data are generated based on the sampled audio data. Since (n-1) pieces of audio data are inserted between them and converted into m pieces of converted audio data, the processing load is reduced by controlling the sampling frequency at the time of audio input by setting the conversion constant. By performing the real-time encoding process accompanying the voice capture, the performance of the encoding device In addition to obtaining quality encoding result of corresponding, due to the sampling data amount is reduced, it becomes possible to reduce the buffer usage for the time data temporary storage.

[0592] According to a speech encoding method of claim 11, in the method of claim 10, in the speech data conversion step, the m / n sampled speech data is converted speech data which is continuous by n pieces each. By creating data, it is possible to easily obtain converted audio data that can reduce the above processing load.

According to a twelfth aspect of the present invention, in the method of any of the seventh to eleventh aspects, the audio buffering step of temporarily holding the sampled audio data in an input buffer; Checking the amount of data in the input buffer, comparing it with a preset value, and executing an input buffer monitoring step of changing the value of the conversion constant n stored in the register based on the result of the comparison. In the input audio sampling step, the sampled audio data is written to the input buffer. In the audio data conversion step, the sampled audio data is read out from the input buffer, and the converted audio data is temporarily stored. Using the data amount as an index, determine the processing capability of the encoding device at that time, By changing the value of the conversion constant n according to the result of, in response to variations in the processing capability of the apparatus, it is possible to perform real-time coding processing with the audio capture.

According to the speech encoding method of claim 13, in the method of any of claims 7 to 11, the amount of encoded data per unit time output in the encoding step is checked. Is compared with a preset value, and based on the result of the comparison, an encoded data monitoring step of changing the value of the conversion constant n stored in the register is performed.
By judging the processing capability of the encoding device at that time and changing the numerical value of the conversion constant n according to the result, the real-time encoding accompanying the voice capture is performed in response to the variation in the processing capability of the encoding device. Can be performed.

[0595] According to the speech encoding method of the present invention, in the speech encoding method for encoding speech using band division encoding, the control constant used for the encoding is stored. A control constant storage step, a sampling step of sampling the input voice and outputting sampling data, and performing band division on the sampling data obtained in the sampling step,
A band dividing step of outputting band signal data; a coding bit allocating step of allocating coded bits to the band signal data obtained in the band dividing step; and Quantizing the signal data, a quantization step of outputting a quantization value, and an encoding step of outputting encoded data based on the quantization value obtained in the quantization step,
Based on the stored control constant, the band division step, the coded bit allocation step, the quantization step, and the encoding process control step of controlling the data processing in the encoding step,
By setting the control constants, it is possible to reduce the processing load, perform the real-time encoding process accompanying the voice capture, and obtain the encoding result of the sound quality according to the performance of the encoding device.

According to a fifteenth aspect of the present invention, in the method of the fourteenth aspect, in the control constant storing step, a unit period determination constant k is stored in the unit period determination constant register as the control constant. The encoding process control step is characterized in that the number of sampling data to be targeted in one band dividing process in the band dividing step is p, and the time corresponding to p pieces of sampling data is a unit period. For each of the p pieces of output sampling data, a determination as to whether the corresponding unit period is a coding target period or a non-coding target period is performed based on the stored unit period determination constant. Is controlled so that the sampling data of the unit period is output to the band division step only when it is determined that the When the period is determined to be the non-encoding target period, in the above-described encoding step, a determination control step is performed in which the previously stored fixed coded data is controlled to be output as coded data. For the delimited sampling data, the encoding process is performed only for the period of the encoding process, and the other encoding process is not performed. It is possible to reduce the amount and perform real-time encoding processing accompanying the audio capture to obtain an encoding result of sound quality according to the performance of the encoding device.

According to the speech encoding method of claim 16, in the method of claim 15, in the determination control step, the i-th unit period is defined as ti, and the stored unit period determination constant k and an arbitrary From the integer n, i = n × k + 1
Holds, it is determined that the unit period ti is the encoding target period, so that the encoding target period is determined according to the setting of the unit period appraisal constant k, and the processing load can be reduced. Become.

[0598] According to the speech encoding method of claim 17, in the method of claim 14, in the control constant storing step, an arithmetic processing determination constant q is stored in the arithmetic processing determination constant register as the control constant. The encoding process control step is included in the band division step, and is based on the stored arithmetic processing determination constant q,
Since this is an arithmetic processing stop step for controlling the arithmetic processing in the band division step to be interrupted halfway, the processing is performed by omitting a part of the arithmetic processing in the band division step according to the setting of the arithmetic processing determination constant q. The burden can be reduced.

According to the speech coding method of the eighteenth aspect, in the method of the seventeenth aspect, in the step of stopping the arithmetic processing, the arithmetic processing of the basic low-pass filter in the band dividing step is performed by using both ends of the filter. By controlling the steps to be terminated halfway, the above-described arithmetic processing can be omitted, and the processing load can be reduced.

According to the speech coding method of claim 19, in the method of claim 14, in the control constant storing step, a band selection constant r is stored in the band selection constant register as the control constant. The encoding process control step includes performing the encoding bit allocation step and the quantum Is a band thinning step for controlling to execute the processing in the
According to the setting of the band selection constant r, it is possible to reduce the processing load by omitting the subsequent processing for a part of the band signal data.

[0601] According to the speech encoding method of claim 20, in the method of claim 19, in the band decimation step, the stored band signal data is output from the M band signal data outputs obtained in the band division step. The band selection constant r
By selecting the band signal data every other, the above-mentioned band signal data is selected and the processing load can be reduced.

[0602] According to the speech coding method of claim 21, in the method of any of claims 14 to 20,
The status of data processing in voice encoding is obtained, and a processing status monitoring step of changing the value of the stored control constant is executed in accordance with the obtained status. By determining the processing capability of the encoding device in the above, and changing the numerical value of the control constant according to the result, the real-time encoding process accompanying the voice capture can be performed in response to the fluctuation in the processing capability of the encoding device. It is possible to do. .

[0603] According to the speech encoding method of claim 22, in the method of claim 21, the processing status monitoring step includes: an audio buffering step of temporarily accumulating sampling data in an input buffer; Compare the amount of data held with a preset value,
By executing the input monitoring step of changing the control constant based on the result of the comparison, the processing capacity of the encoding device at that time is determined using the temporary storage amount as an index, and according to the result, By changing the numerical value of the conversion constant n, it becomes possible to perform a real-time encoding process accompanying voice capture in response to a change in the processing capability of the device.

[0604] According to the speech encoding method of claim 23, in the method of claim 21, the processing status monitoring step includes the step of controlling the amount of the encoded data output per unit time in the encoding step. By comparing with a value set in advance, it is assumed that the encoding monitoring step to change the value of the control constant based on the result of the comparison,
Using the amount of encoded data as an index, the processing capability of the encoding device at that time is determined, and a conversion constant n
By changing the numerical value of, it is possible to perform real-time encoding processing accompanying voice capture in response to fluctuations in the processing capacity of the device.

[0605] According to the speech encoding method of claim 24, the speech encoding method for encoding the original speech information obtained by digitizing the speech by using the band division encoding method. A sampling step of outputting sampling data, performing a band division on the sampling data obtained in the sampling step, and outputting a band signal data; and a sampling step of outputting band signal data. A coded bit allocation step of allocating coded bits to the band signal data, and a bit allocation control step of controlling the allocation in the coded bit allocation step by a psychological auditory analysis alternative control method; and Quantization of the above band signal data according to the allocation of A quantization step of outputting the reduction value,
Based on the quantized value obtained in the above-described quantization step, by executing an encoding step of outputting encoded data, a bit allocation control that simplifies the psychological auditory analysis is executed, and the characteristics of human hearing are performed. It becomes possible to execute an encoding process that can obtain a corresponding high-quality encoding result without greatly increasing the processing load.

[0606] According to the speech encoding method of claim 25, in the method of claim 24, the bit allocation control step is a method of substituting a psychological auditory analysis for the band signal data obtained in the band division step. By adopting a sequential bit allocation step that controls to perform coded bit allocation according to a bit allocation order predetermined by a control method, bits are allocated by a simple algorithm for each band, thereby simplifying psychological hearing. By performing the analysis, it is possible to obtain encoded data with good reproduction sound quality, although the processing load is slightly increased.

[0607] According to the speech coding method of claim 26, in the method of claim 24, the bit allocation control step is a step of substituting the psychological auditory analysis for the band signal data obtained in the band division step. The weighting of each band determined in advance by the control method and the band output adaptive bit allocation step of controlling to perform coding bit allocation based on the output level of each band signal data, The simplified psycho-auditory analysis is performed by allocating bits according to each assignment and the nature of the data itself, and although there is a slight increase in processing load, it is possible to obtain encoded data with good reproduced sound quality. It becomes possible.

[0608] According to the speech encoding method of claim 27, in the method of claim 24, the bit allocation control step includes a step of substituting a psychological auditory analysis for the band signal data obtained in the band division step. An improved band output for controlling weighting of each band predetermined by a control method, weighting of the number of bits allocated to each band, and coding bit allocation based on the output level of each band signal data. By adopting an adaptive bit allocation step, allocation for each band,
According to the nature of the data itself, bits are allocated in consideration of the bit allocation already allocated, and a simplified psycho-auditory analysis is performed. Although there is a slight increase in processing load, encoding with good reproduction sound quality is performed. Data can be obtained.

[0609] According to the speech encoding method of claim 28, in the method of claim 24, the bit allocation control step is performed for each band signal data with respect to the band signal data obtained in the band division step. Is compared with the minimum audible limit value, and bit allocation is not performed for band signal data determined to be less than the minimum audible limit by the above comparison, and control is performed so as to increase bit allocation to other bands. By adopting the limit comparison step, bits are assigned in consideration of the minimum audible limit, and a simplified psycho-auditory analysis is performed. Data can be obtained.

According to the video and audio encoding method of claim 29, when encoding video and audio, some or all of the processing steps included in the two encoding processes are performed by:
In a video / audio coding method executed using a common computer resource, video / audio information composed of original video information composed of a plurality of still image information representing still images per unit time and original audio information representing audio. Is input, an audio buffering step of temporarily accumulating the original audio information, and reading the original audio information accumulated in the audio buffering step, encoding the read original audio information, Using the audio encoding step of outputting the encoded audio information and the encoding load reference information indicating the degree of the encoding load of the video, the processing capability of the video / audio encoding process is determined, and based on the result of the determination, A coding load evaluation step of controlling coding of original video information in a video coding step described later; and A video encoding step of encoding the input still image information constituting the original video information and outputting the encoded video information in accordance with the control performed by the encoding apparatus. It can control video encoding in accordance with the processing capacity of the camera, execute real-time encoding processing accompanying the capture of video and audio, and obtain good encoding results without sound interruption during playback. Become.

[0611] According to the video / audio coding method of claim 30, in the method of claim 29, the coding load evaluation step is performed when the still image information constituting the original video information is input. The total amount of the original audio information stored in the audio buffering step and the encoding load evaluation information are obtained based on the encoding load reference information, and the encoding load evaluation information is compared with a preset load limit. If the encoded load evaluation information does not reach the load limit, still image information is output.If the encoded load evaluation information reaches the load limit, the still image information is discarded. Using the amount of stored audio data as an index, whether to execute or not perform encoding on video data is determined, so that real-time Running-coding processing can be uninterrupted sound reproduction, obtain good coding results.

[0612] According to the video / audio coding method of claim 31, in the method of claim 29, when analog video information is input and video resolution information described later is output,
The analog video information is converted into original video information composed of a plurality of discrete digital pixel information and a plurality of still image information having a resolution according to the video resolution information, and output so as to be processed in the video encoding step. In the encoding load evaluation step, the total amount of the original audio information accumulated in the audio buffering step and encoding load reference information indicating the degree of the image encoding load are calculated. Calculating coding load evaluation information based on the coding load evaluation information, obtaining video resolution information representing a resolution of a video used for video coding, based on the coding load evaluation information, and outputting the video resolution information; In the converting step, when the video resolution information is output, the still image information is matched with the video resolution information in accordance with the video resolution information. And performs encoding processing to output encoded video information.By using the amount of temporarily stored audio data as an index to determine the resolution in encoding of video data, real-time By executing the encoding process, it is possible to obtain a good encoding result without interruption of sound during reproduction without interruption of video.

[0613] According to the video / audio coding method of claim 32, in the method of claim 29, in the coding load evaluation step, the coding load evaluation information is output so as to be processed in the video coding step. To do
In the video encoding step, the still image information is subjected to an encoding process by a processing amount calculated using the output encoding load evaluation information, and is output as encoded video information. By determining the execution ratio of the encoding for the video data using the amount of the reproduced audio data as an index, the real-time encoding process accompanying the capturing of the video and audio is executed, and the sound is not interrupted during reproduction. It is possible to obtain an encoding result without interrupting the video.

[0614] According to the video / audio coding method of claim 33, in the method of any of claims 29 to 32, in the audio coding step, the original audio information accumulated in the audio buffering step is used. Reading, calculating the total amount of the read original audio information and outputting it as a processed audio information amount, and thereafter, encoding the original audio information and outputting it as encoded audio information. In the load evaluation step, an original audio input amount is obtained based on the elapsed time and the input amount of the original audio information per time, and a predicted audio buffer amount which is a difference between the original audio input amount and the processed audio information amount. And the coding load evaluation information is obtained using the predicted audio buffer amount. As a benchmark, by controlling the processing of video data, it is possible to execute real-time encoding processing accompanying the capture of video and audio, and obtain good encoding results without sound interruption during playback. This is possible even when it is impossible or difficult to obtain the amount of voice data.

[0615] According to the video / audio coding method of claim 34, in the method of any one of claims 29 to 32, in the coding load evaluation step, when the still image information is input, an elapsed time is determined. And an input amount per hour of the original audio information, and an amount of processed audio information is obtained based on a total amount of encoded audio information output in the audio encoding step. ,
Further, after calculating a predicted audio buffer amount which is a difference between the obtained original audio input amount and the obtained processed audio information amount, the coding load evaluation information is obtained using the predicted audio buffer amount. By controlling the processing of video data using the predicted buffer amount as an index as an alternative to the amount of temporarily stored audio data, the real-time encoding process accompanying the capture of video and audio is executed, and the sound is interrupted during playback. It is possible to obtain a good coding result without any problem even when it is impossible or difficult to obtain the temporarily stored audio data amount and the processing amount from the coding step.

[0616] According to the video / audio coding method of claim 35, in the method of any one of claims 29 to 34, a change in the result of the judgment in the coding load evaluation step is monitored, and the change is monitored. In response to the above, the above-mentioned encoding load reference information is set, so that by controlling the encoding process for the video, the real-time encoding process accompanying the capture of the video and audio is executed, and the sound is not interrupted during reproduction. In addition to being able to obtain good coding results, by monitoring fluctuations in information used for control and setting control criteria in response to the fluctuations, severe fluctuations in reproduced video image quality can be achieved. Can be suppressed.

[0617] The video encoding apparatus according to claim 36 is a video encoding apparatus for encoding a video, wherein the video encoding is performed on the original video information composed of a plurality of still image information in which the video is digitized. One or more of the image information
Video encoding means for encoding according to encoding parameters to be described later, and intra-frame encoding, forward prediction encoding, backward prediction encoding, and bidirectional prediction encoding using one or more resolutions as one encoding parameter. One or more encoding types among the encoding types including each type of encoding are set as other encoding parameters, and the encoding parameter for determining the processing amount of the encoding means is set to a given frame rate. And a coding parameter determining means for determining the coding parameters based on the coding result. As a result, a good encoding result can be obtained.

[0618] According to the speech encoding apparatus of the present invention, in the speech encoding apparatus for encoding speech by the band division encoding method, the set frequency fs, which is a numerical value used for encoding processing, is used. And a register for storing a conversion constant n, audio input means for inputting audio to be encoded, and sampling audio data generated using the sampling frequency determined based on the stored set frequency fs. Input audio sampling means, and the number of sampled audio data obtained when the set frequency fs is used as the sampling frequency is m, and the number determined based on the conversion constant n is m ′, and m ′
Audio data conversion means for outputting converted audio data consisting of m audio data, including a number of sampled audio data; band dividing means for dividing the converted audio data into bands to obtain M band signals; Set frequency fs stored
Encoding frequency allocating means for allocating coded bits only to a band signal equal to or lower than the restricted frequency out of the band signals, using a frequency fs / 2n obtained from the conversion constant n and A quantization means for performing quantization based on the data, an encoding means for outputting the quantized data as encoded data, and an encoded data recording means for recording the output encoded data. By setting the constant n, it is possible to reduce the processing load, perform the real-time encoding processing accompanying the voice capture, and obtain the encoding result of the sound quality according to the performance of the encoding apparatus.

[0619] According to the speech coding apparatus of claim 38, in a speech coding apparatus for performing coding on a voice using a band division coding method, a control constant used for the coding is stored. A control constant storage unit, a sampling unit that samples input voice and outputs sampling data, a band division unit that performs band division on the sampling data obtained by the sampling unit, and outputs band signal data. Coding band allocating means for allocating coded bits to the band signal data obtained by the band dividing means, and quantizing the band signal data according to the coded bit allocation. A quantizer for outputting a value, an encoder for outputting encoded data based on the quantized value obtained by the quantizer, Based on the stored control constants, the band division means, the coded bit allocation means, the quantization means, and the encoding processing control means for controlling the data processing in the encoding means, the control constant The setting makes it possible to reduce the processing load and to perform a real-time encoding process accompanying the voice capture, thereby obtaining an encoding result of sound quality according to the performance of the encoding device.

[0620] According to the audio coding apparatus of claim 39, in the audio coding apparatus for performing encoding on the audio by using the band division encoding method, the input audio is sampled and the sampling data is obtained. , A band dividing unit that performs band division on the sampling data obtained by the sampling unit, and outputs band signal data, and a band signal data obtained by the band dividing unit. Coded bit allocating means for allocating coded bits, bit allocation control means for controlling the allocation in the coded bit allocating means by a psychological auditory analysis alternative control method, and the band signal data according to the coded bit allocation. And a quantizer for outputting a quantized value, and a quantity obtained by the quantizer. Coding means for outputting coded data based on the coded value, thereby performing bit allocation control that simplifies psycho-auditory analysis and obtaining a high-quality coding result according to the characteristics of human hearing. Encoding processing can be performed without greatly increasing the processing load.

[0621] According to the video and audio encoding apparatus of claim 40, when encoding video and audio, some or all of the processing steps included in the two encoding processes are performed.
In a video / audio coding apparatus executed using a common computer resource, video / audio information composed of original video information composed of a plurality of still image information representing still images per unit time and original audio information representing audio. Is input, audio buffering means for temporarily storing the original audio information,
Audio encoding means for reading the original audio information stored in the audio buffering means, encoding the read original audio information, and outputting encoded audio information, and a code indicating the degree of load of video encoding Using the coding load reference information, determine the processing capability of the video and audio encoding device,
Based on the result of the determination, an encoding load estimating unit that controls the output of the original image information to an image encoding unit described below, and the original image information is configured according to the control of the encoding load estimating unit. And video encoding means for encoding the still image information and outputting encoded video information when the still image information to be input is input, so that the processing capability of the encoding device for performing the video / audio encoding is provided. In response to this, video encoding is controlled, real-time encoding processing is performed along with the capture of video and audio, and it is possible to obtain good encoding results without sound interruption during reproduction.

[0622] According to the video encoding program recording medium of claim 41, in the recording medium in which the video encoding program for encoding the video is recorded, the video encoding program comprises a plurality of still image information in which the video is digitized. For the original video information, a video encoding step of encoding one or more of the still image information according to encoding parameters described below, and one or more resolutions as one encoding parameter;
One or more of the encoding types including intra-frame encoding, forward predictive encoding, backward predictive encoding, and bidirectional predictive encoding are used as the other encoding parameters. Encoding parameters for determining the amount of processing of the encoding step, the encoding parameter determining step of determining based on a given frame rate, and recorded an encoding program to execute, in a general-purpose computer such as a personal computer By executing the encoding program, at a given frame rate, high-resolution or by setting an encoding parameter obtained as an encoding result of a high compression rate, utilizing the device resources, It is possible to obtain a proper encoding result.

[0623] According to the audio encoding program recording medium of claim 42, the audio encoding program is used for encoding processing on an audio encoding program for encoding audio by band division encoding. A storage step of storing a set frequency fs, which is a numerical value, and a conversion constant n, a voice input step of inputting a voice to be encoded, and a sampling frequency determined based on the stored set frequency fs The input voice sampling step of creating the sampled voice data using the set frequency fs as the sampling frequency, and the number of the sampled voice data obtained when the set frequency fs is used as the sampling frequency is m.
An audio data conversion step of outputting converted audio data composed of m audio data, including m ′ sampled audio data, where m ′ is a number determined based on the conversion constant n, where ≧ m ′; The converted audio data is
A band division step of dividing the band to obtain M band signals, and a frequency fs / 2n obtained from the stored set frequency fs and the conversion constant n as a limit frequency. A coded bit allocation step of allocating coded bits only to a signal, a quantization step of performing quantization based on the allocated coded bits, and a coding step of outputting the quantized data as coded data, Since the encoded program for executing the encoded data recording step for recording the encoded data to be output is recorded, the general-purpose computer such as a personal computer executes the encoded program to set the conversion constant n. By
It is possible to reduce the processing load and perform a real-time encoding process accompanying the voice capture, and obtain an encoding result of sound quality according to the performance of the encoding device.

[0624] According to the audio encoding program recording medium of claim 43, the audio encoding program for encoding audio using a band division encoding system is recorded on the recording medium, A control constant storing step of storing a control constant used for sampling, a sampling step of sampling an input voice and outputting sampling data, and performing band division on the sampling data obtained in the above sampling step to obtain band signal data. , A coding bit allocation step of allocating coded bits to the band signal data obtained in the band division step, and allocating the coded bits. A quantization step of performing quantization and outputting a quantization value; Based on the quantized values obtained by the flop,
An encoding step of outputting encoded data; and an encoding step of controlling data processing in the band division step, the encoded bit allocation step, the quantization step, and the encoding step based on the stored control constant. Since the encoding program for executing the processing control step was recorded, the encoding program was executed on a general-purpose computer such as a personal computer, so that the processing load was reduced by setting the control constants and the voice was captured. By performing the real-time encoding process, it is possible to obtain an encoding result of sound quality according to the performance of the encoding device.

[0625] According to the audio coding program recording medium of claim 44, the input audio is recorded on a recording medium in which an audio encoding program for encoding the audio using the band division encoding method is recorded. Sampling process,
Sampling step of outputting sampling data, performing band division on the sampling data obtained in the above sampling step, outputting a band signal data, and performing band division on the band signal data obtained in the above band division step. A coded bit allocation step of allocating coded bits, a bit allocation control step of controlling the allocation in the coded bit allocation step by a psychological auditory analysis alternative control method, and the band allocation according to the coded bit allocation. An encoding program that performs signal data quantization and executes a quantization step of outputting a quantization value and an encoding step of outputting encoded data based on the quantization value obtained in the above quantization step. Because it was recorded, general-purpose personal computers In the computer, by executing the encoding program, performs a bit allocation control that simplifies the psychological auditory analysis, the encoding process of obtaining a high-quality encoding result according to the characteristics of human hearing, Execution can be performed without greatly increasing the processing load.

[0626] According to the video / audio coding program recording medium of claim 45, in coding video and audio, a part or all of the processing steps included in the two coding processes is performed by a common process. In a recording medium on which a video / audio coding program to be executed using computer resources is recorded, the recording medium is composed of original video information composed of a plurality of pieces of still image information representing still images per unit time and original audio information representing audio. When video and audio information is input, an audio buffering step of temporarily storing the original audio information, and reading the original audio information stored in the audio buffering step, and encoding the read original audio information Processing and outputting encoded audio information, and using the encoded load reference information indicating the degree of load of the video encoding, An encoding load evaluation step of controlling encoding of original video information in a video encoding step to be described later based on a result of the determination, and a control in the encoding load evaluation step; And a video encoding step of encoding the still image information constituting the input original video information and outputting the encoded video information, thereby recording the general-purpose computer such as a personal computer. In the above, by executing the encoding program, in accordance with the processing capability of the encoding device that performs the video and audio encoding, the video encoding is controlled, and the real-time encoding process accompanying the capture of the video and audio is performed. By doing so, it is possible to obtain a good encoding result without interruption of sound during reproduction.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a video encoding device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a processing procedure in an encoding unit of the video encoding device according to the embodiment.

FIG. 3 is a flowchart showing a processing procedure in an encoding parameter determination unit of the video encoding device according to the embodiment.

FIG. 4 is a block diagram illustrating a configuration of a video encoding device according to a second embodiment of the present invention.

FIG. 5 is a flowchart showing a processing procedure in a coding parameter determination unit of the video coding apparatus according to the embodiment.

FIG. 6 is a block diagram illustrating a configuration of a video encoding device according to a third embodiment of the present invention.

FIG. 7 is a state transition diagram showing state transitions in an encoding pattern determination unit of the video encoding device according to the embodiment.

FIG. 8 is a block diagram illustrating a configuration of a video encoding device according to a fourth embodiment of the present invention.

FIG. 9 is a state transition diagram showing state transitions in an encoding pattern determination unit of the video encoding device according to the embodiment.

FIG. 10 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 5 of the present invention.

FIG. 11 is a diagram showing a hardware configuration of the apparatus according to the embodiment.

FIG. 12 is a flowchart showing a procedure of speech encoding performed by the apparatus according to the embodiment;

FIG. 13 is a diagram for describing sampling processing and audio data conversion processing by the device of the embodiment.

FIG. 14 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 6 of the present invention.

FIG. 15 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 16 is a block diagram illustrating a configuration of a speech encoding device according to a seventh embodiment of the present invention.

FIG. 17 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 18 is a block diagram illustrating a configuration of a speech coding apparatus according to Embodiment 8 of the present invention.

FIG. 19 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 20 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 9 of the present invention.

FIG. 21 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 22 is a conceptual diagram for describing audio encoding according to the embodiment.

FIG. 23 is a block diagram illustrating a configuration of a speech coding apparatus according to Embodiment 10 of the present invention.

FIG. 24 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 25 is a diagram showing a coefficient Ci in an arithmetic expression of a basic low-pass filter in band division according to the embodiment.

FIG. 26 is a block diagram illustrating a configuration of a speech coding apparatus according to Embodiment 11 of the present invention.

FIG. 27 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 28 is a conceptual diagram for explaining band thinning in speech encoding according to the embodiment.

FIG. 29 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 12 of the present invention.

FIG. 30 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 31 is a block diagram illustrating a configuration of a speech coding apparatus according to Embodiment 13 of the present invention.

FIG. 32 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 33 is a block diagram illustrating a configuration of a speech coding apparatus according to Embodiment 14 of the present invention.

FIG. 34 is a flowchart showing a processing procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 35 is a flowchart showing a processing procedure of sequential bit allocation by the device of the embodiment.

FIG. 36 is a flowchart showing a processing procedure of bit allocation to each band in the sequential bit allocation according to the embodiment.

FIG. 37 is a block diagram illustrating a configuration of a speech encoding device according to Embodiment 15 of the present invention.

FIG. 38 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 39 is a block diagram showing a configuration of a speech coding apparatus according to Embodiment 16 of the present invention.

FIG. 40 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 41 is a flowchart showing a processing procedure of improved band output adaptive bit allocation by the device of the embodiment.

FIG. 42 is a block diagram illustrating a configuration of a speech coding apparatus according to Embodiment 17 of the present invention.

FIG. 43 is a flowchart showing a procedure of speech encoding performed by the apparatus of the embodiment.

FIG. 44 is a diagram illustrating a schematic configuration of a video and audio encoding device according to Embodiment 18 of the present invention.

FIG. 45 is a diagram schematically illustrating an operation of the video and audio encoding device of the embodiment.

FIG. 46 is a diagram for explaining an operation over a longer period of time of the video and audio encoding device of the embodiment.

FIG. 47 is a diagram showing a schematic configuration of a video / audio encoding device according to Embodiment 19 of the present invention.

FIG. 48 is a diagram showing a schematic configuration of a video / audio encoding device according to Embodiment 20 of the present invention.

FIG. 49 is a diagram illustrating a schematic configuration of a video / audio encoding apparatus according to Embodiment 21 of the present invention.

FIG. 50 is a diagram showing a schematic configuration of a video / audio encoding apparatus according to Embodiment 22 of the present invention.

FIG. 51 is a diagram for describing a phenomenon in video / audio coding according to Embodiments 18 to 22;

FIG. 52 is a diagram illustrating a schematic configuration of a video / audio encoding device according to Embodiment 23 of the present invention;

[Fig. 53] Fig. 53 is a diagram for describing video / audio coding according to Embodiment 23.

FIG. 54 is a diagram illustrating a schematic configuration of a video / audio encoding device according to Embodiment 24 of the present invention;

FIG. 55 is a block diagram illustrating a configuration of a video encoding device according to Embodiment 25 of the present invention.

FIG. 56 is a flowchart showing a processing procedure in an encoding unit of the video encoding device of the embodiment.

FIG. 57 is a flowchart showing a processing procedure in an encoding parameter determination unit of the video encoding device of the embodiment.

And FIG. 58 is a block diagram illustrating a configuration of a video encoding device that performs real-time processing and is configured by dedicated hardware according to the related art.

FIG. 59 is a block diagram illustrating a configuration of a speech encoding device according to a first example of the related art.

FIG. 60 is a flowchart showing a processing procedure of speech encoding in the same example.

FIG. 61 is a diagram for describing a sampling process in audio encoding.

FIG. 62 is a conceptual diagram for explaining band division in audio coding.

FIG. 63 is a diagram showing band signals obtained by band division.

FIG. 64: MPEG1Audi according to a second example of the prior art
FIG. 2 is a block diagram illustrating a configuration of an audio encoding device.

FIG. 65 is a flowchart showing a speech encoding process in the same example.

FIG. 66 is applied to a psychological auditory analysis according to the prior art.
It is a figure showing the minimum audible limit in human hearing.

FIG. 67 is a diagram showing a schematic configuration of a video / audio coding apparatus according to a conventional technique.

[Explanation of symbols]

101, 201, 301, 401, 5001 Encoding means 102, 202, 302, 402, 5002 Encoding parameter determination means 103, 203, 303, 403, 5003 DCT
Processing means 104, 204, 304, 404, 5004 Quantization means 105, 205, 305, 405, 5005 Variable length coding means 106, 206, 306, 406, 5006 Bit stream generation means 107, 207, 307, 407, 5007 Inverse quantization 108, 208, 308, 408, 5008 Inverse DC
T processing means 109, 209, 309, 409, 5009 Predicted image generation means 110 Resolution reference table 210 Encoding pattern reference table 310, 410 Encoding pattern determination means 311, 411 Processing capacity determination means 412 Buffer means 413 Input frame rate control means 501,601,701,801,901,1001,
1101, 1201, 1301, 1401, 1501,
1601, 1701, 2551, 2651 Voice input unit 502, 602, 702, 802, 902, 1011
1112, 1202, 1302 registers 810, 1210, 1310 fixed coding registers 503, 603, 703, 803, 903, 1003
1103, 1203, 1303, 1403, 1503
1603, 1703, 2553, 2653 Input audio sampling unit 504, 604, 704, 804 Audio data conversion unit 1118 Band thinning unit 505, 605, 705, 805, 905, 1005
1105, 1205, 1305, 1405, 1505
1605, 1705, 2555, 2655 band division units 506, 606, 706, 806, 906, 1006
1106, 1206, 1306, 1406, 1506,
1606, 1706, 2556, 2656 coded bit allocation units 507, 607, 707, 807, 907, 1007,
1107, 1207, 1307, 1407, 1507,
1607, 1707, 2557, 2657 quantization units 508, 608, 708, 808, 908, 1008,
1108, 1208, 1308, 1408, 1508,
1608, 1708, 2558, 2658 encoders 509, 609, 709, 809, 909, 1009,
1109, 1209, 1309, 1409, 1509,
1609, 1709, 2559, 2659 Encoded data recording unit 7010, 1213 Input buffer 7011, 1214 Input buffer monitoring unit 8012, 1315 Encoded data monitoring unit 1718 Minimum audible limit comparison unit 2660 FFT (fast Fourier transform) unit 2661 Psychological hearing analysis Parts 1801, 1901, 2001, 2101, 201,
2301, 2401, 270 video camera 1802, 1902, 2002, 2102, 2202
2302, 2402, 2702 sound capture unit 1803, 1903, 2003, 2103, 2203
2303, 2403 audio buffering unit 1805, 1905, 2005, 2142, 2205
2305, 2405, 2703 Speech encoder 1806, 1906, 2006, 2106, 2206
2306, 2406, 2704 video capture unit 1807, 1923, 2035, 2107, 2207,
2307, 2407, 2705 video encoders 1808, 1921, 2032, 2144, 2253,
2308, 2408 Encoding load evaluation unit 1924 Inter-frame prediction processing unit 1925 Frame encoding unit 2037 Resolution correction information adding unit 2141, 251, 236, 246 System timer 2411 Encoding load presenting unit 2412 Load setting standard video / audio output unit

 ──────────────────────────────────────────────────続 き Continued on the front page (31) Priority claim number Japanese Patent Application No. 9-42051 (32) Priority date Hei 9 (1997) February 26 (33) Priority claim country Japan (JP) (72) Inventor Hidenori Tatsumi 1-12-20 Hikaricho, Higashi-ku, Hiroshima City, Hiroshima Prefecture Inside Matsushita Electric Industrial Information Systems Hiroshima Research Laboratory (72) Inventor Eiji Kawahara 1-12-20 Hikaricho, Higashi-ku, Hiroshima City, Hiroshima Hiroshima Prefecture Matsushita Electric Industrial Co., Ltd. Inside the Hiroshima Research Institute of Information Systems (72) Inventor Yoshitaka Arase 1-12-20 Hikaricho, Higashi-ku, Hiroshima City, Hiroshima Prefecture Matsushita Electric Information Systems Hiroshima Laboratory

Claims (45)

[Claims] 1. A video encoding method for encoding a video, wherein one or more of the still image information described above is described below with respect to original video information including a plurality of still image information in which the video is digitized. A video encoding step of encoding according to the encoding parameters, a resolution of the original video information, a frame rate required when reproducing encoded data obtained by encoding,
One of a processing performance indicating a processing capability of an encoding device that executes the video encoding step, or one or a plurality of encoding parameters that affects a processing amount of an encoding process in the video encoding step. A coding parameter determining step of determining one or more of the coding parameters based on the above. 2. The video encoding method according to claim 1, wherein the processing capability of the encoding device that executes the video encoding step is output and the determination result is output. A video encoding method, further comprising a determining step. 3. The video encoding method according to claim 1, wherein the encoding parameter indicates a resolution, an intra-frame encoding, or a predictive encoding in an encoding process performed on the original video information. A video encoding method comprising at least one of an encoding type and a detection range for detecting a motion vector used for the predictive encoding. 4. The video encoding method according to claim 2, wherein in the processing capability determination step, the determination is made based on a type of a control device of the video encoding method. Video encoding method. 5. The video encoding method according to claim 2, wherein in the processing capability determination step, the determination is made based on a time required for an encoding process in the encoding step. Video encoding method. 6. The video encoding method according to claim 2, wherein, in the processing capability determining step, the input original video information is temporarily stored, and in the storage, a series of original video information constituting the original video information is stored. A video buffering step of sequentially storing the still image information and sequentially discarding the still image information read out in the encoding step and subjected to the encoding process; and Performing a frame rate control step of controlling the storage of the still image information to be performed at a constant frame rate determined based on the given frame rate, and temporarily storing the image information in the video buffering step. A video code for performing the above-mentioned determination based on the storage amount of the original video information; Method of. 7. A speech encoding method for encoding speech by a band division encoding method, comprising: a storage step of storing a set frequency fs and a conversion constant n, which are numerical values used for an encoding process. An audio input step of inputting audio to be encoded; an input audio sampling step of creating sampling audio data using a sampling frequency determined based on the stored set frequency fs; M is the number of sampled sound data obtained when m is used as the sampling frequency, m is the number determined based on the conversion constant n, and m is the number of sampled sound data including m ′ sampled sound data. An audio data conversion step of outputting converted audio data consisting of: A band division step of obtaining a band signal of the following formula, and using a frequency fs / 2n obtained from the stored set frequency fs and the conversion constant n as a limit frequency, among the band signals, coding bits only for a band signal equal to or lower than the limit frequency. A coding bit allocation step of allocating the following; a quantization step of performing quantization based on the allocated coding bits; an encoding step of outputting the quantized data as encoded data; And a coded data recording step of recording data. 8. The speech encoding method according to claim 7, wherein, in the input speech sampling step, m pieces of sampled speech are processed by sampling the input speech using the stored set frequency fs as a sampling frequency. In the audio data conversion step, sampling audio data is extracted every (n-1) samplings from the m sampling audio data, and two adjacent sampling audio data of the extracted sampling audio data are extracted. A speech encoding method comprising inserting (n-1) pieces of speech data between them and converting them into m pieces of transformed speech data. 9. The audio encoding method according to claim 8, wherein in the audio data converting step, converted audio data in which each of the extracted sampled audio data is continuous by n pieces is created. The audio coding method to use. 10. The speech encoding method according to claim 7, wherein, in the input speech sampling step, a frequency fs obtained from the stored set frequency fs and a conversion constant n.
M / n sampled audio data is created by sampling the input audio using / n as a sampling frequency. In the audio data conversion step, two adjacent audio data are generated based on the sampled audio data. A speech encoding method characterized in that (n-1) speech data is inserted between sampled speech data and converted into m converted speech data. 11. The audio encoding method according to claim 10, wherein, in the audio data conversion step, the m / n sampled audio data creates converted audio data that is continuous by n each. A speech coding method characterized by the above-mentioned. 12. The audio encoding method according to claim 7, wherein an audio buffering step of temporarily holding the sampled audio data in an input buffer, and checking a data amount of the input buffer. Performing an input buffer monitoring step of changing the value of the conversion constant n stored in the register based on the result of the comparison. Writing the sampled audio data into the input buffer; and in the audio data converting step, reading the sampled audio data from the input buffer and converting the sampled audio data. . 13. The speech encoding method according to claim 7, wherein an amount of encoded data per unit time outputted in said encoding step is checked and compared with a preset value. And a coded data monitoring step of changing a value of the conversion constant n stored in the register based on a result of the comparison. 14. A speech encoding method for encoding speech using a band division encoding method, comprising: a control constant storage step for storing a control constant used for the encoding; and a sampling process for input speech. A sampling step of outputting sampling data, a band division step of performing band division on the sampling data obtained in the sampling step, and outputting band signal data, and a band signal data obtained in the band division step. A coded bit allocation step of allocating coded bits, a quantization step of performing quantization of the band signal data according to the coded bit allocation, and outputting a quantized value; An encoding step of outputting encoded data based on the quantized value obtained in the step, Based on the stored control constants, performing the band division step, the coded bit allocation step, the quantization step, and an encoding control step of controlling data processing in the encoding step. The audio coding method to use. 15. The voice encoding method according to claim 14, wherein in the control constant storing step, a unit period determination constant k is stored in the unit period determination constant register as the control constant. In the processing control step, the number of sampling data to be processed in one band dividing process in the band dividing step is p, and the time corresponding to the p pieces of sampling data is defined as a unit period. For each unit, whether the corresponding unit period is the encoding target period or the non-coding target period is determined based on the stored unit period determination constant, and the unit period is the encoding target period and Only when it is determined, control is performed so that the sampling data of the unit period is output to the band division step, and the unit period is set to the encoding pair. A speech encoding method, characterized in that when it is determined that the period is outside the elephant period, the encoding step is a determination control step of controlling to output fixed encoded data stored in advance as encoded data. 16. The speech encoding method according to claim 15, wherein in the determination control step, the stored unit period determination constant k and an arbitrary integer n are set as ith unit periods.
When i = n × k + 1 holds, the unit period t
A speech encoding method, wherein i is determined to be the encoding target period. 17. The speech encoding method according to claim 14, wherein in the control constant storing step,
Storing the arithmetic processing determination constant q in an arithmetic processing determination constant register, wherein the encoding processing control step is included in the band division step, and based on the stored arithmetic processing determination constant q, A speech coding method, characterized in that the speech coding method is a computing process stop step of controlling the computing process to be terminated halfway. 18. The speech encoding method according to claim 17, wherein in the step of stopping the arithmetic processing, the arithmetic processing of the basic low-pass filter in the band division step is aborted halfway for steps at both ends of the filter. A speech encoding method characterized by performing control as described above. 19. The speech encoding method according to claim 14, wherein in the control constant storing step,
Storing a band selection constant r in a band selection constant register, wherein the encoding process control step selects, based on the stored band selection constant r, the band signal data output from the band division step A speech coding method, characterized in that it is a band thinning step for controlling only the coded bit allocation step and the quantization step for only the coded bit allocation step and the quantization step. 20. The speech encoding method according to claim 19, wherein in the band thinning step, r stored band selection constants are obtained from the M band signal data outputs obtained in the band division step. A speech coding method characterized by selecting band signal data every other time. 21. The speech encoding method according to claim 14, wherein a state of data processing in speech encoding is acquired, and the value of the stored control constant is stored in accordance with the acquired situation. And a processing status monitoring step for changing the audio encoding. 22. The audio encoding method according to claim 21, wherein, in the processing status monitoring step, an audio buffering step of temporarily storing sampling data in an input buffer; and an amount of data held in the input buffer. An input monitoring step of comparing with a preset value and changing the control constant based on a result of the comparison. 23. The audio encoding method according to claim 21, wherein the processing status monitoring step includes comparing an amount of the encoded data output per unit time in the encoding step with a preset value. And
A speech encoding method, which is an encoding monitoring step of changing a value of the control constant based on a result of the comparison. 24. A speech encoding method for encoding original speech information obtained by digitizing speech by using a band division encoding method, wherein a sampling process is performed on input speech to output sampling data. A band dividing step of performing band division on the sampling data obtained in the sampling step and outputting band signal data; and encoding bits of the band signal data obtained in the band division step. A coded bit allocation step of allocating, a bit allocation control step of controlling the allocation in the coded bit allocation step by a psychological auditory analysis alternative control method, and quantizing the band signal data according to the coded bit allocation. Performing a quantization step to output a quantization value; A coding step of outputting coded data based on the quantization value obtained in the quantization step. 25. The speech encoding method according to claim 24, wherein the bit allocation control step is performed in advance on the band signal data obtained in the band division step using a psychological auditory analysis alternative control method. A speech coding method, which is a sequential bit allocation step of controlling to perform coding bit allocation according to a bit allocation order. 26. The speech encoding method according to claim 24, wherein the bit allocation control step is performed in advance on the band signal data obtained in the band division step by a psychological auditory analysis alternative control method. A speech encoding method, which is a band output adaptive bit assignment step of controlling to perform encoding bit assignment based on weighting to each band and an output level of each band signal data. 27. The speech encoding method according to claim 24, wherein the bit allocation control step is performed in advance on the band signal data obtained in the band division step using a psychological auditory analysis alternative control method. An improved band output adaptive bit allocation step for controlling to perform coding bit allocation based on weighting for each band, weighting for the number of bit allocations for each band, and output level of each band signal data. A speech encoding method characterized by the following. 28. The speech encoding method according to claim 24, wherein the bit allocation control step includes: determining a minimum audible limit value for each band signal data with respect to the band signal data obtained in the band division step. It is a minimum audible limit comparison step of performing a comparison, performing no bit allocation for band signal data determined to be less than the minimum audible limit by the above comparison, and controlling to increase bit allocation for other bands. Characteristic speech coding method. 29. In encoding video and audio,
In a video / audio coding method in which a part or all of the processing steps included in the above two coding processes are performed using a common computer resource, an original image including a plurality of pieces of still image information representing a still image per unit time is provided. When video and audio information composed of video information and original audio information representing audio is input, an audio buffering step for temporarily storing the original audio information, and an original buffer stored in the audio buffering step. Reading the audio information, encoding the read original audio information, and outputting the encoded audio information; and using the encoding load reference information indicating the load level of the video encoding, Judgment the processing capability of the audio encoding process, and based on the result of the judgment, control the encoding of the original video information in the video encoding step described later. A coding load evaluating step, and a video coding step of coding the still image information constituting the input original video information according to the control in the coding load evaluating step, and outputting the coded video information. A video / audio coding method characterized by being executed. 30. The video / audio coding method according to claim 29, wherein the coding load evaluation step is performed such that when still image information constituting the original video information is input, the still image information is accumulated in the audio buffering step. The coding load evaluation information is obtained based on the total amount of the original audio information and the coding load reference information, and the coding load evaluation information is compared with a preset load limit. Output the still image information when the load limit has not been reached, and discard the still image information when the coded load evaluation information has reached the load limit. Encoding method. 31. The video / audio coding method according to claim 29, wherein analog video information is input, and when video resolution information described later is output, the analog video information is composed of a plurality of discrete digital pixel information. Performing a video capture step of converting the original video information into a plurality of still image information having a resolution according to the video resolution information and outputting the processed original video information to be processed in the video encoding step. In the load evaluation step, the coding load evaluation information is obtained based on the total amount of the original audio information accumulated in the audio buffering step and the coding load reference information indicating the degree of the load of the video coding.
On the basis of the coding load evaluation information, video resolution information representing the resolution of a video used for video coding is obtained, and the video resolution information is output. In the video coding step, the video resolution information is output And a coding process for coding the still image information according to the video resolution information and outputting the coded video information. 32. The video / audio coding method according to claim 29, wherein in the coding load evaluation step, the coding load evaluation information is output so as to be processed in the video coding step. In the encoding step, the still image information is encoded by a processing amount calculated using the outputted encoding load evaluation information, and the encoded image information is output as encoded video information. Video / audio coding method. 33. The video / audio encoding method according to claim 29, wherein in the audio encoding step, the original audio information accumulated in the audio buffering step is read, and the read original audio information is read. The total amount of audio information is calculated and output as a processed audio information amount, and thereafter, the original audio information is coded and output as coded audio information. And an estimated voice buffer amount, which is a difference between the original voice input volume and the processed voice information volume, based on the input volume per hour of the original voice information. A video / audio coding method characterized in that said coding load evaluation information is obtained using a quantity. 34. The video / audio encoding method according to claim 29, wherein, in the encoding load evaluation step, when the still image information is input, an elapsed time and a value of the original audio information The original voice input amount based on the input amount per time; and
A processed audio information amount is obtained based on a total amount of the encoded audio information output in the audio encoding step, and a prediction is a difference between the obtained original audio input amount and the obtained processed audio information amount. A video / audio coding method characterized in that after obtaining an audio buffer amount, the coding load evaluation information is obtained using the predicted audio buffer amount. 35. The video / audio coding method according to claim 29, wherein a change in the result of the determination in the coding load evaluation step is monitored, and the code is provided in response to the change. A video / audio coding method characterized by setting coded load reference information. 36. A video encoding apparatus for encoding a video, wherein one or more of the still image information described below is described with respect to original video information including a plurality of still image information in which the video is digitized. Video encoding means for encoding according to the encoding parameter, and one or more resolutions as one encoding parameter, wherein intra-frame encoding, forward prediction encoding, backward prediction encoding, and bidirectional prediction encoding are performed. Using one or more encoding types among the encoding types including the respective types as other encoding parameters, the encoding parameters for determining the processing amount of the encoding means are determined based on a given frame rate. A video coding apparatus comprising: a coding parameter determining unit for determining. 37. A speech encoding apparatus for encoding speech according to a band division encoding scheme, comprising: a register for storing a set frequency fs and a conversion constant n, which are numerical values used for an encoding process; Voice input means for inputting voice to be encoded; input voice sampling means for generating sampled voice data using a sampling frequency determined based on the stored set frequency fs; When the number of sampled audio data obtained when used as the sampling frequency is m, and the number determined based on the conversion constant n is m ′, m audio data including m ′ sampled audio data are included. Voice data converting means for outputting converted voice data, and converting the converted voice data into M band signals. A band dividing means for assigning coded bits only to a band signal having a frequency equal to or lower than the limit frequency among the band signals, using a frequency fs / 2n obtained from the stored set frequency fs and the conversion constant n as a limit frequency. Bit allocation means, quantization means for performing quantization based on the allocated coded bits, coding means for outputting the quantized data as coded data, and recording the coded data to be output An audio encoding device comprising: encoded data recording means. 38. A speech coding apparatus for coding speech using a band division coding method, comprising: control constant storage means for storing control constants used for the coding; and sampling processing of input speech. Sampling means for outputting sampling data, band dividing means for performing band division on the sampling data obtained by the sampling means, and outputting band signal data, and band signal data obtained by the band dividing means. Against
Coded bit allocating means for allocating coded bits; quantizing means for quantizing the band signal data in accordance with the coded bit allocation and outputting a quantized value; Encoding means for outputting encoded data based on the quantized value obtained, and the band dividing means based on the stored control constants,
A speech encoding apparatus comprising: the encoded bit allocation unit; the quantization unit; and an encoding control unit that controls data processing in the encoding unit. 39. A speech encoding apparatus that encodes speech using a band division encoding method, comprising: a sampling unit that samples input speech and outputs sampling data; Band dividing means for performing band division on the obtained sampling data and outputting band signal data; and for band signal data obtained by the band dividing means,
Coded bit allocating means for allocating coded bits; bit allocation control means for controlling the allocation in the coded bit allocating means by a psychological auditory analysis alternative control method; and Voice, characterized by comprising: a quantizing means for performing quantization of the data and outputting a quantized value; and an encoding means for outputting encoded data based on the quantized value obtained by the quantizing means. Encoding device. 40. In encoding video and audio,
A video / audio coding apparatus that performs part or all of the processing steps included in the above two coding processes using a common computer resource. When video / audio information composed of video information and original audio information representing audio is input, audio buffering means for temporarily storing the original audio information, and audio data stored in the audio buffering means. The audio information is read, the read original audio information is encoded, and audio encoding means for outputting encoded audio information, and the encoding load reference information indicating the load level of the video encoding are used for the video. A coding load estimating unit that controls the output of the original video information to a video coding unit, which will be described later, based on the result of the determination; In accordance with the control of the encoding load evaluator, when still image information constituting the original video information is input, the still image information is encoded, and video encoding means for outputting encoded video information is provided. A video / audio encoding device characterized by the following. 41. A recording medium on which a video encoding program for encoding a video is recorded, wherein one of the still image information and one of the still image information are digitized from the original video information comprising a plurality of still image information. Or a video encoding step of encoding a plurality according to encoding parameters to be described later; and one or more resolutions as one encoding parameter, intra-frame encoding, forward prediction encoding, backward prediction encoding, and An encoding parameter for determining a processing amount of the encoding step is given, with one or more encoding types among encoding types including each type of bidirectional predictive encoding as other encoding parameters. A video encoding program for executing an encoding parameter determining step of determining based on a frame rate determined based on the frame rate. Recording medium. 42. A recording medium on which a speech encoding program for encoding speech by band division encoding is recorded, wherein a set frequency fs and a conversion constant n, which are numerical values used for the encoding process, are recorded. A storage step of storing; a voice input step of inputting a voice to be encoded; and an input voice sampling step of creating sampled voice data using a sampling frequency determined based on the stored set frequency fs. The number of sampled audio data obtained when the set frequency fs is used as the sampling frequency is m, and the number determined based on the conversion constant n, where m ≧ m ′, is m ′, and m ′ Audio data conversion step of outputting converted audio data consisting of m audio data, including sampled audio data A band dividing step of dividing the converted audio data to obtain M band signals, and a frequency fs / 2n obtained from the stored set frequency fs and the conversion constant n as a limiting frequency. A coded bit allocation step of allocating coded bits only to a band signal equal to or lower than the limited frequency; a quantization step of performing quantization based on the allocated coded bits; and the quantized data as coded data. An audio encoding program recording medium, wherein an encoding program for executing an encoding step for outputting and an encoded data recording step for recording the encoded data to be outputted is recorded. 43. A control medium storing a control constant used for encoding, on a recording medium recording an audio encoding program for encoding audio using band division encoding, A sampling step of sampling audio and outputting sampling data; a band division step of performing band division on the sampling data obtained in the sampling step and outputting band signal data; A coded bit allocation step of allocating coded bits to the band signal data obtained, and a quantization step of quantizing the band signal data according to the coded bit allocation and outputting a quantized value And outputs coded data based on the quantization value obtained in the quantization step. Encoding step, based on the stored control constants, the band division step, the encoded bit allocation step, the quantization step, and an encoding processing control step for controlling data processing in the encoding step. An audio encoding program recording medium on which an encoding program to be executed is recorded. 44. A sampling step of sampling an input audio and outputting sampling data on a recording medium having recorded thereon an audio encoding program for encoding the audio using a band division encoding method; Performing band division on the sampled data obtained in the sampling step, and outputting band signal data; and allocating coded bits to the band signal data obtained in the band division step. A coded bit allocation step, a bit allocation control step of controlling the allocation in the coded bit allocation step by a psychological auditory analysis alternative control method, and quantizing the band signal data according to the coded bit allocation. A quantization step for outputting a quantized value; An audio encoding program recording medium characterized by recording an encoding program for executing an encoding step of outputting encoded data based on a quantized value obtained in a decoding step. 45. In encoding video and audio,
A part of or all of the processing steps included in the above two encoding processes may be performed using a common computer resource on a recording medium recording a video / audio encoding program. An audio buffering step of temporarily storing the original audio information when video / audio information composed of original video information composed of image information and original audio information representing audio is input; Reading the stored original audio information, encoding the read original audio information, and outputting encoded audio information; and encoding load reference information indicating a load level of video encoding. To determine the processing capability of the video / audio coding process, and based on the result of the determination, an original video in a video coding step described later. An encoding load evaluation step of controlling encoding of information; and encoding processing of the input still image information constituting the original video information according to the control in the encoding load evaluation step, and outputting encoded video information. A video / audio coding program recording medium, which records a coding program for executing a video coding step.