JP4508029B2 - Encoding device for moving picture information - Google Patents

Description

  The present invention provides a moving image in which effective encoding can be performed without impairing the motion of the moving image when the moving image information is compressed and encoded so as to be transmitted within a predetermined transmission capacity. The present invention relates to an information encoding apparatus.

  In recent years, the use of multimedia in information communication terminals has been rapidly progressing, and it has become an indispensable condition for future business development to produce various added values based on digital data transmission in telephones. In particular, in car phones and PHS (Personal Handy-phone System), transmission of still images and moving images as well as conventional voice and text information has already been studied in earnest. It is expected to be natural in the near future.

  However, when an image is transmitted, if it is transmitted as it is without taking any measures, the amount of TV video data corresponding to about 166 Mbps (mega bit per second) is obtained, so a car phone (9.6 kbps) or If the transmission capacity is about PHS (32 kbps) or the transmission rate of an existing telephone line (about 10 to 30 kbps), it takes a long time to transmit one frame of image data, which is inappropriate for transmission of moving images. It is.

  Corresponding to such a situation, a moving picture compression technique used for a conventional TV conference and a video CD is required, but ITU-T / H. H.261 targets about 64 kbps to 2 Mbps, and ISO / MPEG1 targets about 1.5 Mbps, and cannot be applied to the ultra-low rate transmission of 64 kbps or less.

  On the other hand, H.B. H.263 targets a transmission rate of 64 Kbps or less, but in this case, it targets a wired transmission, and a level of data transmission error (10-2 to 10) occurring in a mobile communication radio system. -3) is not taken. In addition, since the encoders in these standards use motion compensation technology, the amount of calculation increases, and it is difficult to realize cost reduction.

  Furthermore, even within the range of the transmission capacity as described above, when transmitting not only image information but also audio information at the same time, the transmission capacity of the image information is further restricted. In addition, since such voice information is expected to have a large fluctuation in the amount of generation, it is necessary to use it effectively by changing it flexibly even when a transmission capacity is allocated. When the transmission capacity of audio information fluctuates, the transmission capacity of image information also changes accordingly. Under such circumstances, how to effectively encode and transmit moving picture information It becomes a problem.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to efficiently select and encode image information considered necessary for the user within a range in which the transmission capacity is limited, and to obtain high quality moving image information. It is an object of the present invention to provide an image information encoding apparatus that can provide the image data.

  According to the first aspect of the present invention, the change block detection means detects a change block in which the change amount of the image data of the plurality of blocks in the frame with the image data of the corresponding block in the previous frame is equal to or higher than a predetermined level. Since only the detected change block is encoded by the encoding processing means, information about the region where the change occurs within the limited transmission capacity can be transmitted effectively.

Furthermore, since the encoding condition setting means is provided, when encoding the above-described change block, the encoding process condition is determined according to the data such as the generation amount and change amount of the change block within the transmission capacity range. It is possible to transmit a higher quality moving image by selecting useful information by changing the information in the image. In addition, the encoding condition setting means includes a change block determination threshold value, a quantization scale threshold value, and an encoding level indicated in a stepwise manner by a combination of recommended threshold values for the number of change determination blocks for the entire screen. Is determined from a combination of the calculated overall determination level and the buffer amount margin based on the level determination of the motion amount, the color change amount, and the information generation amount, and is determined according to the region in the frame. Therefore, by setting in advance a specific region that is a target for transmitting a moving image with high accuracy, it becomes possible to perform efficient coding processing only on the target region. Since the encoding condition setting means is configured to be able to set areas for different levels of encoding, it is possible to perform efficient encoding processing as described above.

In the invention of claim 2 , since the encoding condition setting means can set the data for setting the area in a predetermined format, a code is added in that format when the corresponding area is specified. Thus, the encoding process can be performed without increasing the amount of information.

In the invention of claim 3, the encoding condition setting unit, when setting a plurality of regions, a priority corresponding to the height of importance for allocation of the amount of information when the encoding processing for each region Since it can be set, the encoding level can be preferentially performed in descending order of priority according to the degree of occurrence of the information amount, and efficient encoding processing according to the user's request is performed. Will be able to.

In the invention of claim 4, the encoding condition setting means, so is provided to be set different encoding conditions for each region when a plurality sets an area, required in accordance with the state of the screen Information can be efficiently encoded.

In the invention of claim 5 , when the change amount of the image data of the entire frame is reduced by the encoding condition setting means, the determination threshold value used when detecting the change block by the change block detection means is changed to be low. Because it is set, when the number of changed blocks is large or when the amount of information is large, the threshold is increased to control the number of detected changed blocks to be reduced. On the condition that the margin is increased, the change block can be detected following the small change. As a result, for example, even when a situation occurs in which a part of a region to be moved in the frame is stuck, this can be resolved and a natural screen can be provided.

In the invention of claim 9 , when the change amount of the image data of the entire frame is reduced by the encoding condition setting means, the quantization scale used for encoding the change block by the change block detection means is changed to be low. Because it is set, when the number of changed blocks is large or when the amount of information is large, the threshold is increased to control the number of detected changed blocks to be reduced. On the condition that the margin is increased, the quantization scale can be set finely so as to follow even a small movement. As a result, for example, even when a situation occurs in which a part of a region to be moved in the frame is stuck, this can be resolved and a natural screen can be provided.

In the inventions of claims 10, 11 and 12 , the encoding condition setting means changes or determines the encoding processing condition for the invariant area of the image data in the frame, the entire screen or the specific area in the above case. Change settings that lower the threshold value or change settings that lower the quantization scale, so use this period with a high margin to update the invariant area, improve or identify the movement of the entire screen. The movement of the area can be improved.

In the thirteenth aspect of the invention, the encoding condition setting means changes and sets the frame size as necessary from the relationship between the amount of image information generated in a frame and the transmission capacity that can be transmitted. Encoding processing can be performed with an appropriate frame size.

According to the fourteenth aspect of the present invention, when the frame size is reduced and set by the encoding processing means, the image data of the frame before the change is sampled so as to correspond to the image data of the frame after the change. Therefore, it is not necessary to perform a new intra-frame encoding process at the time of changing the frame size, and the change can be made without increasing the information generation amount.

According to the fifteenth aspect of the present invention, when the encoding processing means changes the frame size to the enlargement side, the interpolation filter is used when the image data of the frame before the change corresponds to the image data of the frame after the change. Since the image data is generated by interpolating the image data of the adjacent pixels with respect to the pixel having no image data, it is not necessary to perform a new intra-frame encoding process at the time of changing the frame size, and the amount of information generation is increased. You will be able to make changes without

In the invention of claim 16 , since the encoding function setting means can set a protection function for invalidating the input for changing the encoding condition from the outside, the user can set the protection function by setting the protection function. It is possible to reject an encoding process mode that is not desired by the user, and to prevent encoding from being performed more than necessary from the outside, thereby protecting privacy.

In the invention of claim 17 , the encoding condition setting means stores a mode condition for setting the incidental situation with a predetermined code in accordance with the subject forming the frame or the application. When encoding is performed using the mode condition during processing, a code for specifying the mode condition is added to the transmission signal, so that it is possible to easily set the incidental status and to set the mode condition. The amount of information for the encoding process can be reduced.

In the invention of claim 18 , 19 or 20 , the audio information processing means encodes the audio information and transmits it simultaneously with the image information. At this time, the image information is encoded by the encoding processing means. Since the image transmission capacity at the time of transmission is calculated by subtracting the audio transmission capacity allocated to the transmission of audio information from the total transmission capacity, encoding processing is performed within that range, so that the transmission of audio information depends on the level of image information By assigning the remaining transmission capacity to the image transmission capacity without being hindered, the image information can be transmitted by efficiently using the transmission capacity.

According to the twenty-first aspect of the present invention, the ratio of the voice transmission capacity to the total transmission capacity can be set by the capacity ratio setting means, whereby the encoding condition setting means sets the ratio by the capacity ratio setting means. The voice transmission capacity is set based on the ratio, and the transmission control of voice information and image information according to the user's request can be performed.

In the invention of claim 22 , when the ratio of the voice transmission capacity to the whole transmission capacity is specified by the encoding condition setting means from another device that transmits and receives the transmission signal, the voice transmission capacity is set based on this. In the case of monitoring audio information or image information from the outside, it is possible to efficiently obtain information according to the user's request.

In the invention of claim 23, the sampling time set to be short enough to identify the impulse noise within the allowable delay time set so as not to hinder voice communication by the voice information processing means is set. When the audio signal input within the sampling time is integrated as a unit and the integrated value exceeds the threshold for audio increase judgment, the audio transmission capacity is set within the preset transmission capacity limit. Since it is set to be large, it can be transmitted in a sensitive manner to the state of generation of audio information, and when the amount of generated audio information is small, the transmission capacity of image information can be increased.

In the invention of claim 24 , since a plurality of types of setting patterns of encoding processing block groups in which a plurality of blocks in a frame are set as one group are registered in advance by the encoding processing means, the transmission signal is encoded. Image information can be efficiently transmitted simply by adding information on a setting pattern when processing is performed. Also, at this time, by selecting the coding processing block group pattern set in order from the center in the frame, or the one set sequentially from the part corresponding to the specific area, the first part of the transmission signal The block data of the main part of the frame can be transmitted, and the image information can be transmitted efficiently.

An embodiment in which the present invention is applied to a wireless communication apparatus that transmits audio information and image information simultaneously will be described below with reference to the drawings.
The configuration of this embodiment will be described with reference to FIG. First, in the encoder 1 functioning as a transmitter, a camera 2 as an imaging unit captures the upper body of the user and outputs image information as RGB analog signals. The A / D converter 3 converts an analog signal given from the camera 2 into a digital signal and outputs it. The RGB / CIF conversion unit 4 converts the RGB digital signal given from the A / D conversion unit 3 into a CIF (Common Intermediate Format) signal and outputs it.

  A two-dimensional high-speed DCT (discrete cosine transform) unit 5 receives a CIF signal from the RGB / CIF conversion unit 4 and performs a two-dimensional DCT (H. (Two-dimensional orthogonal transformation specified in H.261) is processed and a signal of DCT coefficient is output. The change area extraction unit 6 serving as a change block detection unit is configured to receive a CIF signal from the RGB / CIF conversion unit 4, and detects a change block based on the CIF signal as described later. In accordance with the result, a change block to be subjected to the two-dimensional DCT process is designated for the two-dimensional high-speed DCT unit 5.

  The quantization unit 7 has a quantization scale QSC set by an encoding control unit 8 as an encoding condition setting unit, and a quantization scale in which a signal of a DCT coefficient given from the two-dimensional high-speed DCT unit 5 is set. The QSC performs linear quantization conversion or dead zone added quantization conversion processing, and, for example, quantizes with an accuracy of 9 bits and outputs the result. The significant coefficient attribute control unit 9 performs attribute control on the quantized DCT coefficient signal provided from the quantization unit 7 in accordance with the condition provided from the coding control unit 8 and outputs the signal to the variable length coding unit 10. To do.

  The variable length encoding unit 10 assigns a corresponding variable length code to the DCT coefficient signal and converts the signal through the layer combining unit 11 to generate a bit stream based on the syntax, and outputs the bit stream to the buffer 12. The buffer 12 controls the output speed of data output to the signal transmission unit 13 and outputs the internal data amount to the encoding control unit 8. The signal transmission unit 13 includes an RS232c communication processing unit 14 and a digital cellular phone 15, and transmits the bit stream signal output from the buffer 12 via the digital cellular phone 15 by radio. The digital cellular phone 15 is set to a transmission capacity (transmission rate) of 9600 bps (bit per second), for example, and an audio signal and an image signal are transmitted within the range of the transmission capacity.

  Next, the configuration of the decoder 16 that functions as a receiver will be described. The signal receiving unit 17 includes a digital mobile phone 18 and an RS232c communication processing unit 19. The bit stream signal received by the digital mobile phone 18 is received by the input buffer 20. The parser 21 takes in data stored in the buffer 20, performs syntax analysis, and outputs it to the variable length decoding unit 22. The variable length decoding unit 22 decodes the variable length code of the input signal and outputs it to the inverse quantization unit 24 via the significant coefficient attribute reproduction unit 23.

The inverse quantization unit 24 inversely converts the reconstructed 9-bit quantized output into a 12-bit signal and outputs it. Further, the two-dimensional high-speed inverse DCT 25 restores 8-bit pixel data by performing a two-dimensional inverse DCT process. The decoding control unit 26 controls the inverse quantization unit 24 based on the coding attribute decoded by the significant coefficient attribute reproduction unit 23.
The CIF / RGB conversion unit 27 converts 8-bit pixel data from the CIF format into an RGB signal, converts it into an analog signal via the D / A conversion unit 28, and outputs the analog signal to the display device 29.
The configuration of the decoder 16 is configured to execute two functions of a variable GOB pattern and an adaptive change of the image data size. However, when this function is not performed on the encoder 1 side. Can adopt the same configuration of the decoder 16 as that of the existing standard.

  Next, the operation of the present embodiment will be described first with reference to FIGS. 2 to 6, and further with reference to FIG. For convenience of explanation, the following items are described separately. (1) Outline description of principle of encoding process, (2) Overall flow of encoding process (see FIGS. 2 to 6), (3) Detailed description of processing contents of each part.

(1) Outline explanation of principle of encoding process In the H.261 and MPEG standards, interframe coding is used, in which two-dimensional DCT (two-dimensional discrete cosine transform) is performed on the interframe difference of block pixel values in the spatial domain. Therefore, at the time of decoding, this difference data is decoded and accumulated between frames to reconstruct an image. However, in this method, when a bit error occurs in the digital wireless communication path, 261 and H.264. In a protocol with a complicated syntax such as H.263 or MPEG standard, the value of each codeword is completely different from the original value, and an error is propagated in the time direction.

  Therefore, in this embodiment, the data compression degree cannot be expected as much as the case of encoding a general inter-frame difference used in the wired system, but the intra-frame coding (intra code) in which the error is completed within the frame. By adopting a configuration that applies only a block) on a block basis, the wireless system is made strong. That is, for the first frame, intra DCT encoding, which is intra-frame encoding, is performed for all blocks, and in the encoding processing for the second and subsequent frames, “changed” is determined by detecting a changed block described later. Intra DCT encoding is performed only on the changed blocks, that is, changed blocks.

  In this method, since the inter-frame difference is not encoded, a loop calculation for obtaining the motion compensation difference in the H261 standard becomes unnecessary. Therefore, this eliminates the need for a configuration for motion vector detection processing and inverse quantization processing, and eliminates the need for a loop filter and a prediction memory for storing pixel data for one frame. As a result, the main calculation contents are only (a) two-dimensional DCT processing, (b) quantization processing, (c) variable length coding processing, (d) buffer control, and (e) change block detection processing, As a result, it is possible to achieve a functional block configuration of a level that can operate in real time with a configuration using software of a personal computer.

(2) Description of overall flow of encoding process (FIGS. 2 to 6)
The encoder 1 encodes the image information as follows according to the flowcharts showing the schematic flow shown in FIGS. First, the encoder 1 clears the variable PIC indicating the number of frames to zero (step S1), takes in the image information captured by the camera 2 (step S2), and converts it into a digital signal in the A / D conversion unit 3. After that, the RGB / CIF conversion unit 4 performs RGB / CIF conversion (step S4).

  Subsequently, the encoder 1 performs step S5 (see FIG. 3) for observing the state of the image information, step S6 (see FIG. 4) for determining the encoding state and request, and step S7 (see FIG. 5) for determining the encoding parameter. ) And step S8 (see FIG. 6) of the encoding calculation process are sequentially executed, and are repeatedly executed until the frame number LPIC reaches a predetermined number NPIC (step S9). The contents of steps S5 to S9 will be described below with reference to FIGS.

(A) Image signal state observation (see FIG. 3)
When the encoder 1 proceeds to step S5 for observing the state of the image information, the encoder 1 sets a GOB pattern according to the feature of the scene according to the program (see FIG. 3) (step A1). Next, in the case of the second frame and thereafter, the encoder 1 calculates the number of changed blocks based on the calculation result in the previous frame before performing the encoding calculation of the current frame (step A3). Then, the level determination of the color change amount, the motion amount, and the information generation amount of the entire image frame is performed (steps A4 to A6).
Such level determination will be described in detail separately. In addition, since the first frame after the scene changes is forcibly subjected to intra DCT coding for all blocks, the processing of steps A4 to A6 is not performed.

  Based on the definition of the inter-block change amount in each frame, the luminance change amount and the color change amount are calculated in units of MBK (macroblock) (steps A11 and A12), and the change set by the default value or the previous frame calculation is determined. The change block is determined based on the threshold value (step A13). If it is not a change block, the process jumps to step D8 of the encoding operation process of step S8 shown in FIG. 6. If it is determined that the block is a change block, the change block number counter is incremented and this program is terminated. .

(B) Determination of encoding state and request (see FIG. 4)
The number of bits (buffer amount BF) of the encoded bit stream stored in the buffer 12 of the encoder 1 is checked (step B1). Next, the encoder 1 determines the setting state (ON / OFF) of the protection request from the user (step B2). Further, the request degree Ra regarding the AV ratio is determined based on the request values of the partner terminal and the self terminal (step B3). Next, the state of the speech signal is detected (step B4), and the speech coding rate BRA is determined according to the state of the detected speech signal such as a silent state or sudden speech (step B5). Next, the data rate margin ADR and the target image coding rate BRV0 are determined from the values of the buffer amount BF of the buffer 12 and the audio coding rate BRA (step B6).

  On the other hand, a total level judgment value L of the amount of change between frames of the image is calculated from each of the level judgment value LM of the motion amount SM, the level judgment value LC of the color change amount DC, and the level judgment value LI of the information generation amount. (Step B7), a recommended value of the encoding parameter is calculated with reference to a table (see Table 1 described later) from the level determination value LADR of the data rate margin ADR and the overall level determination value L (Step B8). .

(C) Determination of coding parameters (see FIG. 5)
Next, the encoding parameter is determined. Here, based on the values of the audio coding rate BRA and the target image coding rate BRV0, the image coding rate BRV is calculated from the coding rate BR which is the overall transmission capacity (step C1). In this case, between these values,
BRA + BRV ≦ BR
The value of the image coding rate BRV is determined under this condition, and the data rate margin ADR is determined after monitoring the buffer amount BF. Therefore, the image coding rate BRV is slightly modified according to the value of the data rate margin ADR. This is because, for example, if the buffer amount BF is smaller than the average sufficient amount BFa, the data rate margin ADR increases accordingly, so that the image encoding rate BRV can be set large.
The frame rate and the image size are determined with reference to the table (see Table 3) from the overall determination level L, the data rate margin ADR, and the image requirement RV (steps C2 and C3). Further, the quantization scale QSC is determined with reference to the above table (step C4).

  Next, the determination threshold value NCB_TH for the number of changed blocks of the entire screen controls the number of transmissions of the DCT significant coefficient, which is determined with reference to Table 1 (step C5). Similarly, the changed block determination threshold value DTH is determined with reference to Table 1 (step C6). Further, the sampling density for calculating the change amount between blocks is determined from the level determination result of the current value of the CPU load of the entire system (step C7).

(D) Encoding operation (see FIG. 6)
The encoding operation is performed based on the encoding parameters determined in the above (A) to (C). This is performed by the method of 261/263 (steps D1 to D14). In this embodiment, no special processing is included except that the MBK attribute of the changed block is forcibly set to intra (intraframe coding). That is, two-dimensional DCT processing, quantization processing, significant coefficient control processing, and variable length encoding processing are sequentially performed (steps D3 to D6), followed by MBK attribute control processing, MBK attribute encoding processing, and GOB attribute encoding processing. , PIC attribute encoding processing is sequentially performed (steps D8 to D13), and then bitstream hierarchical combination is performed (step D14).

(3) Detailed Description of Processing Contents of Each Part Next, the processing contents of each part taken up in the description of the overall flow of the encoding process described above will be described in detail. The description items are as follows.

[A] Changed block detection [a-1] Calculation of change between blocks [a-2] Threshold determination [b] Coding and communication status determination [b-1] Data rate margin [b- 2] Determination of motion amount [b-3] Determination of color change amount [b-4] Determination of information generation amount [c] Control for reducing information generation amount [c-1] Determination of change amount of entire screen Control of transmission coefficient [c-2] Change determination threshold control based on coding state [c-3] Control according to region [d] Adaptive change of image data size [e] Between audio data and image data [E-1] Continuous control [e-2] Protection function [e-3] Priority order for AV ratio control [e-4] Mode control [f] Parameter control [f-1] Request Determination of coding rate based on degree [f-2] Code of remote monitoring mode Of remote control parameters [f-3] System-wide parameter control [g] removing block noise [h] Variable GOB transmission structure identification number [a] detection of a change block

How much the macroblock at a certain address has changed with respect to the previous frame is determined as follows.
[A-1] Calculation of inter-block change amount (a) Inter-block change amount With respect to a macroblock (hereinafter referred to as MBK) input as an original image, an absolute value of an inter-frame error with MBK one frame before The sum is calculated for each MBK. Here, since MBK is composed of six BLKs (blocks), the total change amount E (n, m), luminance change amount EL (n, m), and color change amount EC (n, m) are expressed as follows. Can be defined.

here,
E (n, m): the total change amount from the previous frame for the mth MBK of the nth frame
EL (n, m) indicating luminance change between the previous frame and the mth MBK of the nth frame
EC (n, m); color change between previous frame and mth MBK of the nth frame
Value indicating quantity MB (n, m, k); k-th BLK of m-th MBK in n-th frame
d (BLKi, BLKj); an error function between BLKi and BLKj. Here pixel unit
Calculate the sum of absolute differences.

BLKi; i-th block pixel α; weight coefficient of error of color information (a) Simplification of calculation In the above-described calculation of d (BLKi, BLKj), the difference is not calculated for all the pixels in the block. As shown in FIG. 7, only the sampled pixels can be subject to change amount calculation. FIG. 4A shows a case where calculation is performed for all pixels (8 × 8 = 64) in BLK, and FIG. 4B shows that every other pixel in BLK is sampled (4 × 8 = 4). 32 shows a case of 1/2 of all pixels), and FIG. 10C shows sampling every other pixel in the BLK every other row (4 × 4 = 16, 1/4 of all pixels). Shows the case.

(C) Simple motion vector detection Since the amount of calculation of the block matching method can be reduced by sampling the pixels in the block in the same manner as in (a) above, the amount of calculation of motion vector detection can also be reduced.

(D) Execution of motion detection limited to the change amount area / gaze area If the encoding target area can be limited to a specific area by applying signal change conditions, model conditions, gaze conditions, etc. By performing motion detection only within a region, the amount of computation can be reduced.

[A-2] Threshold Determination Here, E (n, m) is determined by a certain threshold DTH, and for example, it is determined whether or not there has been a change with respect to the previous frame as follows.

case 1) When E (n, m) ≧ DTH → It is determined that there has been a change, and intra coding is started.
case2) When E (n, m) <DTH → It is determined that there is no change, and encoding is not performed.

  As described above, for example, according to an experiment, when the data rate is 9.6 kbps, the quantization scale parameter QSCALE is set to 20 (quantization step = 40) and the change detection threshold value DTH is set to 2000 for the CIF image. By setting it to a degree, relatively good semi-moving image transmission can be performed.

[B] Coding and communication state determination [b-1] Data rate margin ADR
(A) Definition In ultra-low-rate communication, there is not much room in communication data rate (transmission capacity) when coding moving objects such as human images, but there are no people present or remote monitoring. In some cases, the change does not occur so much that the data rate can be afforded. Such a data rate margin ADR is defined as follows.

ADR = (image communication rate) − (encoded information amount of previous frame)
This can be substituted by detecting the buffer margin corresponding to the increase / decrease in the buffer amount BF of the encoder output buffer 12.

(A) Dynamic application control of image communication rate The image communication rate (image transmission capacity) can be changed from moment to moment in accordance with the degree of AV ratio required and the current amount of voice communication data. Therefore, when it is determined that there is little audio communication data, a mode is set in which the communication rate initially set can be changed according to the request level to increase the image communication rate.

(C) Prompt response to sudden increase in audio data When the audio communication rate (audio transmission capacity) is reduced by the method as described above, the image encoder 1 can be used to respond quickly to sudden increase in audio data. The output is stopped, the transmission of the image encoder output buffer 12 is stopped, and the AV rate setting is returned to the initial value (the value determined from the request level). For detection of increase in voice data, a time width that is sufficiently distinguishable from impulse noise is set as a sufficiently small time T with respect to the maximum delay time TAdelay that does not interfere with voice communication, and a voice signal within the time T is set. The integrated value IA is calculated to be equal to or greater than a certain threshold value, and it is determined that the voice data is increased.

(D) Explanation of operation flowchart In the above-described adaptive control of the image data rate with respect to the increase / decrease in the amount of generated audio data in (a) and (c), the control is performed according to the flowchart of the program shown in FIG.
First, it is detected whether or not there is a sudden increase in audio data (steps T1 and T2). Here, as described above, the integration value IA of the audio signal within the sampling time T is calculated (step T1), and a significant change occurs when the integration value IA exceeds a certain threshold value. As a result, it is determined that the voice data is increased (step T2).

  Next, when an increase in audio data is not detected from the determination result, that is, in a silent state, the audio encoding rate is reduced (step T5), and the image encoding rate is increased instead (step T6). On the other hand, when sudden increase in the audio data is detected, the calculation of the image encoding is stopped (step T7) and the transmission of the image data is stopped (step T8). The image conversion rate and image encoding rate (AV rate) are returned to the initial values set based on the user's request level (step T9). Further, when it is not a silent state but is not a sudden increase in speech data, the speech encoding rate is not changed and the previous state is maintained.

[B-2] Determination of Motion Amount Regarding the determination of the motion amount, a method for calculating the total motion amount in the following two cases will be described. Note that the difference between the case where motion detection is not performed (a) and the case where motion detection is performed (B) is that the motion amount can be determined more accurately, but the amount of calculation due to motion detection is large. In terms of increase, these can adopt a method of performing motion detection in accordance with the processing capability of the CPU constituting the control unit when it does not become a heavy burden when processing in real time.

(A) When motion detection is not performed The center of gravity calculation of the changed block is performed, and the size of the motion vector for the previous frame is set as the center of gravity motion vector vg. A value obtained by multiplying the center-of-gravity motion vector vg by the number of change blocks NCB is defined as a total motion amount SM. These are expressed as follows:

here,
NMB: number of MBKs in the frame IG (n): horizontal component of the centroid coordinate of the target area in the nth frame JG (n): vertical component of the centroid coordinate of the target area in the nth frame (a) When performing motion detection The magnitudes of motion vectors for the previous frame of all macroblocks are calculated, and the total sum of the magnitudes of the obtained motion vectors is calculated and used as the total motion amount SM.

上述のようにして得られる総動き量ＳＭから、あらかじめ設定されている複数のしきい値を用いて複数段階のレベルに判定してレベル判定値ＬＭを得る。この動き量のレベル判定値ＬＭは後述する符号化制御において用いられるようになっている。

From the total motion amount SM obtained as described above, a level determination value LM is obtained by determining a plurality of levels using a plurality of preset threshold values. This motion level judgment value LM is used in the encoding control described later.

[B-3] Determination of Color Change Amount Of the change amounts calculated in the change block detection, the change amounts for only the color blocks are totaled over the entire screen to calculate the total color change amount DC. This is shown as:

here,
DC (n); total color change amount in the nth frame NMBK; number of MBKs in one frame (22 × 18 = 396 in the CIF format image)
Is)
EC (n, m); color change from previous frame for m-th MBK of n-th frame
Quantity This DC (n) is level-determined by some set threshold values, and this is set as a level judgment value LC.

[B-4] Determination of information generation amount The level of the information generation amount of the entire screen one frame before is determined based on some set threshold values. This determination result is set as a level determination value LI. LI is defined by the following causal relationship.
RA → [Rv, ADR] → LI
This is because, by setting the audio request level Ra, the image request level Rv is determined with respect to the capacity of the entire coding rate, and the information allowed from the relationship between the image request level Rv and the data rate margin ADR. A generation level determination value LI is determined.

[C] Control for reducing the amount of information generated [c-1] Determination of change amount of entire screen and control of transmission coefficient Based on the number of changed blocks obtained as a result of the threshold determination described above, Determine the degree of change. Using this, for example, the transmission of DCT coefficients is controlled as follows.

case1) When NCB ≧ NCB_TH1 → Transmit only the DC component of the DCT significant coefficient of the changed block
case2) When NCB_TH1> NCB ≧ NCB_TH2 → Change blocks other than the gaze area transmit only the DC component
case3) When NCB_TH2> NCB → Transmit all significant coefficients of change block However,
NCB: Number of changed blocks in one screen NCB_THi (i = 1, 2,...); Judgment threshold (variable depending on data rate margin and usage mode)
It is.

[C-2] Change Determination Threshold Control Based on Coding State The change region determination threshold is determined from the motion amount determination level LM, the color change amount determination level LC, the information generation amount determination level LI, and the data rate margin determination level LADR. The value DTH and the quantization scale QSC are determined dynamically. That is,
[LM, LC, LI, LADR] → [DTH, QSC, NCB_TH]
And In this case, for the LM, LC, and LI among the level determination values, specifically, for example, the overall determination level L is calculated using the following equation, and the determination is performed based on the result.

Therefore, the change area determination threshold value DTH, the quantization scale QSC, and the entire screen change determination threshold value NCB_TH are determined from the combination of the overall determination level L and the data rate margin LADR. Therefore,
[L, LADR] → [DTH, QSC, NCB_TH]
When L and LADR take values in four stages of 0 to 3, respectively, a control example as shown in Table 1 below can be adopted.

[C-3] Control according to area (a) Area setting
<1> Automatic region setting Multiple regions are set by methods such as gaze region, motion and color clustering. Several methods are conceivable for this method, but they are not adopted in the present embodiment, so only the possibility is described and not mentioned here.

<2> Manual setting For example, there are various setting methods as shown below, and either or both of them can be used for setting.

i) A method for setting a scene to be transmitted while the user watches a monitor image on the transmission side.
ii) A method in which the user sets the scene on the transmission side while viewing the received image on the reception side.

(A) Description of a plurality of areas The plurality of areas set above are described by their centroids and area widths. For example, when a rectangular area parallel to the line direction is described, it is as shown in Table 2 below. Here, a CIF format macroblock is described as a unit.

  In the above case, the area category can be determined by human intervention in the manual setting as described above. Also, some methods for automatic setting are conceivable, but are omitted here.

(C) Area monitoring
<1> Fixed area monitoring Once the center of gravity and size of the area are determined, it can be monitored as a two-dimensional fixed window. In this case, the image area in the window can be reproduced as a moving image with high image quality and high update speed by preferentially allocating the information amount.

<2> Tracking of moving area Rather than fixing the initially set area centroid and size, the movement of the object reflected in the area by the method used for automatic setting and frame correlation (for example, motion detection) You can track based on. This can be realized by a moving object region extraction method used for model-based three-dimensional motion estimation for modeling and registering a three-dimensional shape of an object or a person and predicting what is estimated for the movement.

<3> Transmission of region information Encoding processing focused on the required region by transmitting only the region information defined above (region number, center of gravity, horizontal width, vertical width) and image information within that region. Thus, the amount of information generated can be reduced efficiently.

<4> Area priority When multiple areas are specified, priorities are assigned when allocating information. Here, the area numbers are assigned in order from the areas considered to be important for image transmission. However, the background area number is 0. For priority change, only the flag indicating change and change information are sent. For example, the change information is sent by setting the order with numbers corresponding to a plurality of areas.

(3421) → In this case, it indicates that area 3 is transmitted with the highest priority, area 4 is transmitted second, area 2 is transmitted third, and area 1 is transmitted fourth.
(D) Area-specific threshold setting The change determination threshold value DTH is automatically set according to the priority. For example, in the order of DTH0 to DTH3,
[1000, 1500, 2000, 2500]
Set as follows. Alternatively, the change determination threshold value DTH for each region can be set individually.

(E) Adaptive change of setting threshold When a sudden change in image content (scene change, appearance / disappearance of moving objects, etc.) ends and the amount of change in the entire screen decreases, the data rate margin ADR also increases. . At this time, in order to improve the image quality, the following methods can be considered.
<1> Decreasing the change threshold DTH for the invariant region <Updating the invariant region>
<2> Decrease the change threshold DTH for the entire screen <Improve the movement of the entire screen>
<3> Lowering the change determination threshold DTH only for the gaze area <improving the movement of the gaze portion>

  Particularly, <1> is a countermeasure for an image deterioration phenomenon that occurs when the change determination threshold value DTH is excessively increased in advance. For example, when a hand is held in front of the camera, the entire screen is switched to flesh color once, but after that, when there is a block that does not exceed the change determination threshold DTH even if the hand is moved, the background such as a wall A situation occurs in which a part of the hand remains as an invariant region.

  For <2>, if the value is lowered too much, the number of changed blocks increases rapidly due to a slight luminance unevenness or hue change, which causes a phenomenon that the frame rate is lowered.

On the other hand, for the quantization scale QSC, the following method can be considered.
<1> Lowering the quantization scale QSC of the invariant region <Improvement of image quality in the invariant region>
<2> Lower the quantization scale QSC of the entire screen <Improvement of image quality of the entire screen>
<3> Lowering the quantization scale QSC only for the gaze area <Improving the image quality of the gaze portion>
FIG. 9 is a flowchart showing the flow of the above control contents. Here, the setting conditions for the change determination threshold values DTH0 to DTH3 and the quantization scales QSC0 to QSC3 are as follows.

<1> Data rate margin ADR increases DTH0 ≧ DTH3 ≧ DTH2 ≧ DTH1
QSC0 ≧ QSC3 ≧ QSC2 ≧ QSC1
<2> Data rate margin ADR increases DTH3 ≧ DTH2 ≧ DTH1 ≧ DTH0
QSC3 ≧ QSC2 ≧ QSC1 ≧ QSC0

The encoder 1 monitors whether or not the data rate margin ADR has changed in a state where the change determination threshold value DTH and the initial values DTH0 and QSC0 of the quantization scale QSC are set (step P1). P2). When the value of the data rate margin ADR changes, when the region-specific adaptation is not performed (step P3), the change determination threshold value DTH and the quantum are referred to by referring to the table (Table 1) as described above. The change scale QSC is changed and set (step P4).
Further, when performing area-specific adaptation, it is determined whether or not it is the gaze area priority mode (step P5). If “YES”, the gaze area change determination threshold value DTH is changed to DTH1. (Step P6), the quantization scale QSC is changed to QSC1 (Step P7).

  When the gaze area priority mode is set, the gaze area change determination threshold is set to DTH2, the invariant display area change determination threshold is set to DTH3 (steps P8 and P9), and gaze is set. The quantization scale of the area is set to QSC2, and the quantization scale of the invariant display area is set to QSC3 (steps P10 and P11). Thereafter, depending on whether or not there is a significant change block exceeding the minimum value DTHmin of the change determination threshold value (step P12), in the case of “YES”, the change determination threshold value of the significant change block is set to DTH2. If not, the process ends.

[D] Adaptive change of image data size In the same manner as described above, in addition to the comprehensive determination of the determination level LADR of the motion amount determination level LM, the color change amount determination level LC, the number of changed blocks NCB, and the data rate margin ADR, The image data size can be adaptively changed in consideration of the level LRV (described later) of the image request level RV.
Here, it is assumed that the image data size can be set for QCIF and SQCIF in addition to the normal CIF. And for example,
[L, LADR, LRV] → [SQCIF / QCIF / CIF]
When L, LADR, and LRV take values in four stages from 0 to 3, the control contents can be set as shown in Table 3 below.

When the image data size is changed as described above, it is possible to continuously change the size between frames without using an intra frame obtained by performing intra encoding on the frame immediately after the change. Both enlargement change and reduction change can be realized as follows.
(A) CIF → QCIF
By driving CIF / QCIF downsampling, the CIF encoded frame (stored in the frame memory) at time t (n−1) is converted into QCIF. QCIF-based interframe coding at time t (n) is performed on this QCIF. On the decoder side, downsampling is driven by the QCIF instruction flag in the picture header.

(B) QCIF → CIF
By driving the QCIF / CIF bilinear interpolation filter, the QCIF encoded frame (stored in the frame memory) at time t (n−1) is converted into CIF. CIF-based interframe coding at time t (n) is performed on this CIF. On the decoder side, the interpolation filter is driven by the CIF instruction flag in the picture header. Some interpolation filters are recommended for switching between sizes.
With the above method, the size can be changed without interposing an intra frame, so that the frame rate can be maintained without extremely increasing the amount of information on the encoder 1 side. Become.

[E] Extraction of degree of requirement for ratio of audio data and image data In order to simplify the human interface, when expressing the demand for communication quality, the minimum required control input means is one dimension shown below Consider in scale. That is, as the required degree Ra for the voice ratio,
Raij: Indicates the degree of request for the speech ratio of the encoder of the terminal j set by the terminal i.

[E-1] Continuous Control As shown in FIG. 10, a knob 29 for designating a ratio (AV ratio) between audio (A) and video (V) is provided on the front surface of the display device 29. This is provided so that the ratio can be set by a slide operation by the user. Now, when the value T indicated by the knob 29a changes from 0 to 1,
Raij = T; ratio of audio data in the entire data Rvij = 1−T; ratio of video data in the entire data. This
<1> Mode control
<2> Change detection threshold (DTH)
<3> Change of quantization scale (QSCALE)
I do.

[E-2] Protect function In order to protect privacy in communication, a protect function that always requires the user's permission when operating the transmission quality of the terminal (terminal 1) on the other terminal (terminal 2). Is provided. More specifically, as shown in FIG. 11, the protection is realized by ON / OFF protection using a mode setting switch. In the normal dialogue mode, when the protection is OFF, the manual mode for manually setting various audio and image parameters and the request of both the own terminal (terminal 1) and the partner terminal (terminal 2) as shown in the next section There is an automatic setting mode that is determined based on the degree. Further, for the above-mentioned purpose, when the protection is ON, no request set from the partner terminal (terminal 2) is accepted.

[E-3] Priority Order for AV Ratio Control When the above protection function is turned off, in the case of one-to-one communication, the following two requests are generated in the terminal 1 from the definition of Raij. Will compete.
Ra11: Degree of request for speech encoder of own terminal (terminal 1) Ra21: Degree of request generated for speech encoder of own terminal 1 from partner terminal (terminal 2) Therefore, it is presumed that the controllability of the user for background information is lower for video information than for audio information, so it is necessary to improve the controllability when video is sent from your terminal compared to audio The following decision rule is set.
RA1 = max [Ra11, Ra21]
RA1: Final request to be sent to the speech coder of terminal i This also corresponds to the fact that it is essential to satisfy the requirements for the voice quality of the listener in order to satisfy the telephone function as a communication at a minimum. ing.

[E-4] Mode Control Mode control according to the subject, application, etc. is performed in advance between the encoder 1 and the decoder 16 according to the use environment, subject, application, etc. as an incidental state of the image, for example. The determined coding conditions are provided, and the amount of information to be generated can be greatly reduced by adding the codes attached to the respective modes to the bit stream data.

Here, the following modes can be considered as settable modes. For example, the modes corresponding to the use environment include indoor, outdoor, or in a car, and in those environments, a fixed mode or a movable mode. The subject mode includes a person mode, a landscape mode, a stationary object mode, a drawing mode, and a text mode. Further, in the human mode, various modes such as the head, the upper body, the whole body, and a plurality of modes can be considered.
Furthermore, there are various modes related to encoding control, such as image center priority mode, target area priority mode, motion priority mode, quasi-video mode, still image mode, and model-based prediction mode. An example is only outlined.

[F] Parameter Control [f-1] Determination of Encoding Rate Based on Request Level Determination of Encoding Rate Based on Request Level The encoding rate is determined from the image request level and the data rate margin. That is,
[RV, ADR] → [BRA, BRV]
BRA: Audio coding rate BRV: Image coding rate Based on this target, the amount of image data is adjusted by changing the quantization scale (QSCALE) and the change detection threshold (DTH) using the methods already described. And the AV ratio is controlled.

[F-2] Remote control of encoding parameter in remote monitoring mode Similar to the case of determining the voice request level described above, priority is given to the control value for its own terminal when setting each value of the encoding parameter. Such a protection function is provided.
In such a remote monitoring mode, the following effects can be obtained by turning on the protection function as described above. That is, in the normal remote monitoring mode, when the protect function is turned off, a set (area) of a fine encoding control parameter transmitted from the partner terminal (terminal 2) on its own terminal (terminal 1) side. Information, quantization scale, various thresholds, etc.), but setting the protect function to ON will reject acceptance of these parameter sets, That is, the change setting is prohibited, that is, the monitoring is continued while maintaining the encoding to be performed under the same conditions.

[F-3] Parameter Control of Entire System The above parameter determination relationships can be summarized as shown in FIG. That is, as a factor for determining the encoding parameter, there is an information amount generated by an image, which includes a motion amount SM, a color change amount DC, a change block number NBC, an information generation amount LI one frame before, a buffer amount (data There are factors such as rate margin (ADR), and the information generation level related to the image is taken into consideration from the result of determining these levels. The rate BRV is determined, and this is determined by associating the encoding parameters of the frame at that time.

As described above, the coding parameter elements to be determined include the image coding rate BRV, the quantization scale QSC, the change block determination threshold value DTH, the image data size CIF / QCIF / SQCIF, and the sampling density of the change amount between blocks. SD, threshold value NCB_TH of the number of change determination blocks for the entire screen, and the like.
Then, calculation processing is performed based on the encoding parameters determined in this way. In addition, such a coding state is constantly observed, and is used when determining the coding parameter from the next time.

[G] Removal of Block Noise A post filter is used as shown in FIG. 13 to remove block noise. This is because when decoding is performed on the decoder 16 side and an image is reproduced, the luminance and color between adjacent blocks are controlled by controlling the transmission level of the DCT coefficient when reproducing in block units. This is to remove or mitigate the influence of signal fluctuations as noise.

  Actually, for example, BLK noise removal processing is performed using a post filter along a line between MBKs (macroblocks) reproduced as shown in FIG. In this BLK noise removal processing, as shown in FIG. 5B, the pixel data arranged along the BLK boundary is weighted using the data of eight adjacent pixels and added. Is converted so that the pixel data becomes intermediate with the data of surrounding pixels. In this case, for example, conversion processing is performed by weighting and adding own pixel data, 4 adjacent pixel data, 2 diagonally adjacent pixel data at a ratio of 1.

[H] Transmission of variable GOB structure and identification number Both a pre-defined common GOB pattern and its ID code are stored in both the encoder 1 and the decoder 16, and the GOB pattern used in the encoder 1 is stored. Are added to the compressed image data and transmitted using the user data area in the existing protocol (H.261, H.263, MPEG, etc.).

  As the variable GOB pattern, for example, those shown in FIGS. 14 and 15 are conceivable. That is, in FIGS. H.261 standard, H.264. The GOB pattern according to the H.263 standard is shown, and the modified GOB pattern is adopted in FIGS. 15 (c), (d) and FIG. 15 (a).

  The modified GOB (part 1) is a pattern in which 4 × 4 macroblocks are grouped to form one GOB unit and arranged on the screen. In the modified GOB (part 2), the macroblock is arranged in the center of the screen 4 × 8. In addition to the arrangement, macroblocks adjacent to the outer periphery thereof are sequentially arranged as a unit of GOB. In the modified GOB (part 3), the modified GOB divided into a predetermined pattern is arranged. In FIG. 15 (b), the designated area is set as GOB, and in FIGS. 15 (c) and (d), a GOB pattern in which QCIF and SQCIF are embedded is arranged.

  Now, by setting the GOB pattern in this way, the decoder 16 decompresses the transmitted compressed bit stream in units of GOB using a normal image decoding protocol, and then stores the GOB stored in the user data area or the like. The corresponding GOB pattern is selected from the GOB pattern database based on the pattern ID code, and the decoded GOB data is arranged based on the GOB number decided in advance and the number of macroblocks in the GOB. By this method, for example, compared with a uniform GOB structure by horizontal scanning from the upper left part to the lower right like a conventional image, priority between GOBs according to image contents can be given.

  For example, in the “modified GOB pattern No. 2” (see FIG. 14D), since the transmission is performed first from the center portion at the center of the screen, an important image portion can be obtained even when an error is mixed in the transmission line and retransmission is repeated. Only can be sent fast with statistically little delay time. In addition, in the “deformed GOB pattern 3” (see FIG. 15A), since the GOB can be divided into a shape that is almost similar to a front image of a person, efficient code allocation in units of GOB is possible. I have to.

“QCIF embedding” (see (c) in the same figure) corresponds to the case where an important part is to be viewed as a moving image with good image quality even if the code rate assigned to the image within the transmission rate is reduced. Thus, it becomes possible to easily shift to the encoding of only the central QCIF. Note that when a plurality of designated areas are GOB, the pattern identification number alone is not sufficient, so use the area description and transmission method described in the control section according to the area of [c-3] described above. Is required.
Here, there arises a problem of how to classify the background image portion outside the designated area as a GOB. The rule corresponding to such a case is the same between the encoder 1 and the decoder 16. If the contents are defined and stored, it is not necessary to transmit new segmentation information.

According to such a present Example, the following effects can be acquired.
That is, first, the change region extraction unit 6 detects a change block among the macroblocks in the frame and performs intra-frame coding processing only on the change block. It is possible to effectively transmit information about a region where a change occurs within the range of transmission capacity (for example, 9600 bps), to perform image reproduction that is resistant to transmission errors, and to perform strong moving image transmission even in a wireless system. The

Furthermore, since intra-frame coding is performed, unlike the case of general video compression, it is not necessary to calculate a difference with respect to the previous frame or a calculation loop of motion compensation differential coding, and the amount of calculation processing is correspondingly increased. For example, it is possible to sufficiently cope with processing by a personal computer or the like, and image transmission using a portable information terminal can be performed.
In addition, when encoding the change block by the encoding control unit 8 as the encoding condition setting means, the encoding processing condition is set according to the data such as the generation amount and the change amount of the change block. It becomes possible to transmit a higher quality moving image by selecting necessary information by changing within the range.

  Second, the encoding control unit 8 determines the amount of information generated by the changed block from the amount of motion, the amount of color change, the number of changed blocks, the margin of the data rate, the speech encoding rate, and the like. Therefore, it is possible to change and set the encoding condition flexibly according to the change of the image.

  Third, based on the change in the amount of information generation and the margin of the data rate as described above, the encoding parameters are the image encoding rate, the quantization scale, the change block determination threshold, the image data size, and the inter-block Since the sampling density for the change amount calculation or the threshold value for the number of blocks for the change determination of the entire screen is changed, effective encoding can be performed during the encoding process.

Fourth, when audio information is transmitted simultaneously, the transmission capacity of image information is obtained according to the state of occurrence of the audio information, and the encoding conditions are adaptively changed and set. Thus, it is possible to efficiently transmit the image information and accurately reproduce the area required by the user.
Fifth, since the sudden increase in audio information is detected and the transmission of image information is elastically limited, image information can be transmitted with high accuracy when there is little audio information while giving priority to the transmission of audio information. become able to.
Sixth, since it is possible to set the required degree of encoding of audio information, it is possible to transmit image information while effectively using the image information within the range of transmission capacity while giving priority to the audio information over the image information. .
Seventh, since a protect function is provided, it is possible to protect privacy against setting of the degree of encoding request from the outside.

The present invention is not limited to the above embodiment, and can be modified or expanded as follows.
The video source is not limited to the camera 2 but may be video source image information.
The medium for transmitting and receiving signals at the signal transmission unit and the signal reception unit is not limited to a digital cellular phone, and other communication means may be used. Furthermore, the present invention is applicable not only to a wireless system but also to a wired communication path. You can also.

The present invention is not limited to the case where audio information and image information are transmitted simultaneously, but can also be applied to the case where only image information is communicated.
The AV ratio setting knob may be a rotary type or digitally set in addition to the slide type. Moreover, it can also be set as the structure omitted as needed.
A protection function may be provided as necessary.
The reference table for determining the encoding rate can be set as appropriate in addition to Tables 1 to 3.
Control of adaptive change of the variable GOB pattern and adaptive change of the image data size can be provided as necessary. Further, when these are not used, the existing standard can be applied in the configuration of the decoder.
The case of a 9600 bps mobile phone has been described. The present invention can be similarly applied to a transmission capacity of about 64 kbps or less, which is targeted by the H.263 standard, and can be effectively used for moving image transmission.

1 is an overall block diagram showing an embodiment of the present invention. Flow chart showing the overall operation Flow chart showing image information state observation operation Flowchart showing encoding state and request determination operation Flowchart showing encoding parameter determination operation Flowchart showing encoding operation processing operation Explanatory diagram showing pixel sampling pattern Flow chart showing detection operation of variation in audio data amount Flow chart showing operation of threshold value change control according to region Front view of display device showing knob portion for setting AV ratio Action explanatory diagram showing the operation of the protect function Block diagram showing the overall relationship of coding parameter determination Illustration of post filter Variable GOB pattern (1) Variable GOB pattern (2)

Explanation of symbols

  1 is an encoder, 2 is a camera (imaging means), 3 is an A / D conversion unit, 4 is an RGB / CIF conversion unit, 5 is a two-dimensional high-speed DCT unit, and 6 is a change area extraction unit (change block detection means). 7 is a quantization unit, 8 is an encoding control unit (encoding condition setting means), 9 is a significant coefficient attribute control unit, 10 is a variable length encoding unit, 11 is a layer combination unit, 12 is a buffer, and 13 is a signal. Transmitter, 14 is an RS232c communication processor, 15 is a digital mobile phone, 16 is a decoder, 17 is a signal receiver, 18 is a digital mobile phone, 19 is an RS232c communication processor, 20 is a buffer, 21 is a parser, 22 Is a variable length decoding unit, 23 is a significant coefficient attribute reproduction unit, 24 is an inverse quantization unit, 25 is a two-dimensional fast inverse DCT unit, 26 is a decoding control unit, 27 is a CIF / RGB conversion unit, and 28 is a D / D A conversion unit, 29 is a display, 29a is AV It is a knob for rate setting.

Claims (24)

In a video information encoding apparatus for generating a transmission signal within a predetermined transmission capacity by performing a process of compressing one frame of image data composed of a plurality of blocks while encoding each block,
Change block detection means for detecting when the amount of change from the image data of the previous frame among a plurality of blocks in the frame is equal to or greater than a determination threshold, as a change block;
When encoding the image data of the frame within the transmission capacity range that can be transmitted, the amount of motion obtained as the information amount of the change block detected by the change block detection means, the color change amount, the number of change blocks, The amount of information generated one frame before, the image coding rate, the quantization scale, the changed block determination threshold, the image data size, the sampling density for calculating the amount of change between blocks, depending on the value of the buffer amount, Encoding condition setting means for changing and setting the threshold value of the number of change determination blocks for the entire screen;
Coding processing means for generating a transmission signal by performing coding processing of intra-frame coding only on the changed block under the coding processing conditions set by the coding condition setting means,
The encoding condition setting means sets the encoding level indicated stepwise by a combination of a change block determination threshold, a quantization scale threshold, and a recommended threshold for the number of change determination blocks for the entire screen. A total determination level is calculated by determining the level of the motion amount, the color change amount, and the information generation amount, and is obtained from a combination of the calculated total determination level and the margin of the buffer amount, and according to the region in the frame An apparatus for encoding moving image information, characterized in that the encoding level can be set differently and an area where the encoding level is different can be set. The apparatus for encoding moving image information according to claim 1,
The apparatus for encoding moving image information, wherein the encoding condition setting means is provided so that data for setting the area can be set in a predetermined format. The apparatus for encoding moving image information according to claim 1 or 2,
The encoding condition setting means is provided such that when a plurality of the areas are set, priority can be set according to the degree of importance of information amount allocation when encoding is performed on each area. An apparatus for encoding moving image information, comprising: The apparatus for encoding moving image information according to claim 3,
The apparatus for encoding moving image information, wherein the encoding condition setting means is provided so that different encoding conditions can be set for each region when a plurality of the regions are set. The apparatus for encoding moving image information according to any one of claims 1 to 4,
The encoding condition setting unit is configured to change and set a determination threshold value used when detecting a change block by the change block detection unit when the change amount of the image data of the entire frame decreases. An apparatus for encoding moving image information. The apparatus for encoding moving image information according to claim 5 ,
The moving picture characterized in that the encoding condition setting means performs the change setting of the encoding processing condition or the change setting for lowering the threshold value of the determination for an invariant region of image data in a frame. Image information encoding apparatus. The apparatus for encoding moving image information according to claim 5 ,
The moving image characterized in that the encoding condition setting means performs the change setting of the encoding processing condition or the change setting for lowering the threshold value of the determination for the entire screen of image data in a frame. Image information encoding apparatus. The apparatus for encoding moving image information according to claim 5 ,
The moving image characterized in that the encoding condition setting means performs the change setting of the encoding processing condition or the change setting for lowering the threshold value of the determination for a specific area of image data in a frame. Image information encoding apparatus. The apparatus for encoding moving image information according to any one of claims 1 to 5 ,
The encoding condition setting means is configured to change and set the quantization scale used when encoding the change block by the change block detection means when the change amount of the image data of the entire frame decreases. An apparatus for encoding moving image information. The apparatus for encoding moving image information according to claim 9 ,
The encoding condition setting unit is configured to change the encoding process condition, or to change the setting for lowering the determination threshold, or to lower the quantization scale for an invariant region of image data in a frame. An apparatus for encoding moving image information, wherein the change setting is performed . The apparatus for encoding moving image information according to claim 9 or 10,
The encoding condition setting unit is configured to change the encoding processing condition, the change setting for lowering the determination threshold, or the quantization scale for the entire screen of image data in a frame. An apparatus for encoding moving image information, wherein the change setting is performed . The apparatus for encoding moving image information according to any one of claims 9 to 11 ,
The encoding condition setting means, for a particular area of the image data in the frame, the change setting of the encoding process conditions, or the change setting to lower the threshold of the determination, or lower the quantization scale An apparatus for encoding moving image information, wherein the change setting is performed . The apparatus for encoding moving image information according to any one of claims 1 to 12,
The video information is characterized in that the encoding condition setting means is configured to change and set the frame size as necessary from the relationship between the amount of image information generated in the frame and the transmittable transmission capacity. Encoding device. The apparatus for encoding moving image information according to claim 13 ,
It said encoding means, said when modified set to the side to reduce the frame size, configured to fit by sampling so as to correspond to the image data of the frame after changing the image data of the pre-change frame An apparatus for encoding moving image information, characterized in that: The apparatus for encoding moving image information according to claim 13 or 14 ,
When the encoding processing means is set to change to enlarge the frame size, when the image data of the frame before the change is made to correspond to the image data of the frame after the change, the pixel adjacent to the pixel without the image data An apparatus for encoding moving image information, comprising: an interpolation filter for generating image data by interpolating from the image data . The apparatus for encoding moving image information according to any one of claims 1 to 15 ,
The apparatus for encoding moving image information, wherein the encoding condition setting means is configured to be able to set a protection function for invalidating an input for changing and setting an encoding condition from the outside . The apparatus for encoding moving image information according to any one of claims 1 to 16 ,
The encoding condition setting means stores a mode condition for setting the incidental situation with a predetermined code according to the subject forming the frame or the application, and the mode condition is set at the time of the encoding process. An apparatus for encoding moving picture information, characterized in that a code for specifying a mode condition is added to a transmission signal when encoding is performed using the. The apparatus for encoding moving image information according to any one of claims 1 to 17 ,
Audio information processing means for encoding audio information to be transmitted simultaneously with image information;
The encoding processing means performs encoding processing within the range by subtracting the audio transmission capacity allocated to the transmission of the audio information from the transmission capacity, as the image transmission capacity when encoding and transmitting the image information. An apparatus for encoding moving image information, characterized in that the apparatus is configured to perform . The apparatus for encoding moving image information according to claim 18 ,
The apparatus for encoding moving image information, wherein the encoding condition setting means is configured to determine the image transmission capacity relative to the transmission capacity based on the audio transmission capacity . In the coding apparatus of the video information according to claim 18 or 19, wherein,
The coding condition setting means is provided so that the voice transmission capacity can be set in advance, and sets the coding condition in the range of the set voice transmission capacity according to the generation of voice information. An apparatus for encoding moving picture information, wherein the encoding condition is set by increasing the image transmission capacity when the audio transmission capacity is lower than the audio transmission capacity . The apparatus for encoding moving image information according to claim 20 ,
Providing a capacity ratio setting means for setting a ratio of the voice transmission capacity to the transmission capacity;
The apparatus for encoding moving picture information, wherein the encoding condition setting means sets an audio transmission capacity based on the ratio set by the capacity ratio setting means . The apparatus for encoding moving image information according to claim 20 or 21,
The encoding apparatus for moving picture information, wherein the encoding condition setting means sets an audio transmission capacity based on a ratio of an audio transmission capacity to a transmission capacity designated by another apparatus that transmits and receives the transmission signal . The apparatus for encoding moving image information according to any one of claims 18 to 22,
The voice information processing means has a sampling time set in a unit of a sampling time set to be short enough to identify impulse noise within a range of an allowable delay time set to a level that does not hinder voice communication. The voice transmission capacity is set to be large within the range of the preset transmission capacity limit when the voice signal input at is integrated and the integrated value exceeds the threshold for voice increase judgment. An apparatus for encoding moving image information, comprising: The apparatus for encoding moving image information according to any one of claims 1 to 23,
The encoding processing means has registered in advance a plurality of types of setting patterns of encoding processing block groups in which a plurality of blocks in a frame are set as one group, and is set when encoding processing is performed on a transmission signal. An apparatus for encoding moving image information, characterized by adding pattern information.

Images