WO2017104011A1

WO2017104011A1 - Image coding apparatus

Info

Publication number: WO2017104011A1
Application number: PCT/JP2015/085146
Authority: WO
Inventors: 亞矢子根本
Original assignee: 三菱電機株式会社
Priority date: 2015-12-16
Filing date: 2015-12-16
Publication date: 2017-06-22

Abstract

In order to obtain an image coding apparatus with which it is possible to improve the image quality of an important region without exceeding a predetermined bit rate, this image coding apparatus is characterized by being provided with: a frame memory unit that sorts input image signals from input order into coding order; an image coding unit that codes the image signals output from the frame memory unit using intra-frame prediction, inter-frame prediction, and variable-length coding; a transmission buffer unit that accumulates data coded by the image coding unit and transmits the data at a predetermined bit rate; and a specific-region rate control unit to which information regarding a specific region is input, that controls quantization steps inside and outside the specific region so that the coded data falls within the predetermined bit rate, and that transmits information regarding the quantization steps to the image coding unit.

Description

Image encoding device

The present invention relates to an image encoding device that encodes an image taken by a camera.

In recent years, a surveillance camera system (CCTV: Closed Circuit) that captures a surveillance area from a camera, distributes it to an image recording and display device, divides and displays a plurality of camera images, and sequentially compresses and records image data on a recording medium. TeleVision system) is widely used. In the surveillance camera system, various signal processing is applied to an image taken by a camera, and moving image data is compressed with high efficiency. Examples of encoding techniques for compression include MPEG-2, H.264, and the like, which are international standards. H.264 / AVC, H.H. 265 / HEVC and the like. These are hybrid coding schemes combining motion compensation prediction coding, discrete cosine transform coding, variable length coding, and the like. Motion compensation prediction includes intra-frame prediction (intra prediction) and inter-frame prediction (inter prediction). The type of picture to be encoded is classified according to the method of motion compensation prediction, and only encoding by intra-frame prediction is performed. Forward prediction in which motion prediction is performed between an intra-coded picture (hereinafter referred to as an I picture) and a temporally forward frame, and the difference value and motion prediction information are transmitted. Bidirectional predictive coded picture (hereinafter referred to as B picture) which performs motion prediction between a coded picture (hereinafter referred to as P picture) and temporally forward, backward or bidirectional frames and compresses data. (Referred to as pictures). In general, encoding is performed by periodically repeating a group of pictures called GOP (Group Of Pictures) starting from an I picture by using these motion compensated prediction encodings.

When photographing a predetermined area with a surveillance camera, the image quality should be reduced for important areas of particular interest, such as where a person's face is reflected at the entrance of a condominium, or where a person's face or money is reflected near a cash register in a store. There is a desire to improve and improve visibility.

In a conventional image encoding device, in order to improve the image quality of a region of interest, whether each region is a motion region or a static region is determined based on the magnitude and difference value of a motion vector between an input frame and a reference frame for motion prediction. Performs smoothing filter processing by determining a non-moving part such as a background part as a static area, and determining a moving part such as a person part as a moving area and skips the smoothing filter process to perform encoding processing. As a result, the image quality of the motion region has been improved (for example, Patent Document 1).

JP 2005-295215 A

In a conventional image encoding device, a large amount of code may be required depending on the magnitude of motion and the area of the motion region, and it is not possible to suppress the amount of code only by performing smoothing filter processing on the still region, or the motion region There was a problem that did not become high image quality.

The present invention has been made to solve the above-described problems. In a surveillance system, moving image data is compressed by inter-frame and intra-frame encoding, and encoded data is transmitted at a predetermined bit rate in real time. An object of the present invention is to provide an image encoding device that can encode a specific area in a screen with high image quality.

An image encoding device according to the present invention is as follows.
A frame memory unit that rearranges input image signals from the input order to the encoding order, and an image code that encodes the image signals output from the frame memory unit using intra-frame prediction, inter-frame prediction, and variable-length encoding And
A transmission buffer unit that accumulates data encoded by the image encoding unit and transmits the data according to a predetermined bit rate;
Information on the specific area is input, and quantization steps inside and outside the specific area are controlled so that the encoded data can be within the predetermined bit rate, and the information on the quantization step is used as the image encoding unit. A specific area rate control unit to transmit to,
It is characterized by comprising.

According to the present invention, it is possible to obtain an image encoding device capable of improving the image quality of an important area without exceeding a predetermined bit rate.

It is a block diagram which shows the whole structure of the image coding apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the hardware constitutions of the image coding apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the write-in and read-out picture in the image coding apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the example of specific area information in the image coding apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the process flow of the quantization step control in the specific area control part of the image coding apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the whole structure of the image coding apparatus which concerns on Embodiment 2 of this invention. It is a block diagram which shows the whole structure of the image coding apparatus which concerns on Embodiment 3 of this invention. It is a block diagram which shows the whole structure of the image coding apparatus which concerns on Embodiment 4 of this invention. It is a block diagram which shows the whole structure of the image coding apparatus which concerns on Embodiment 5 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the overall configuration of an image coding apparatus according to Embodiment 1 of the present invention.
In FIG. 1, 1 is a camera unit, 2 is a frame memory unit, 3 is an image encoding unit, 4 is a transmission buffer unit, 5 is a rate control unit, and 6 is a specific area control unit. The rate control unit 5 and the specific region control unit 6 constitute a specific region rate control unit.

In FIG. 1, a camera unit 1 converts a captured image into a digital signal, performs various signal processing, and outputs an image signal 101. A frame memory unit 2 includes a memory that stores the image signal 101 in units of frames. The block and image encoding unit 3 reads the image signal 103 in accordance with the frame memory read control signal 102 generated so as to rearrange the image signal 101 stored in the frame memory unit 2 in the encoding order, and performs intra-frame prediction or inter-frame prediction. The transmission buffer unit 4 temporarily stores the video stream data 104 output from the image encoding unit 3 and generates a read control signal 107 generated according to the bit rate 106. In accordance with the video stream data 105 and the video stream data The rate controller 5 outputs a picture type 110 and a quantization step 111 based on the bit rate 106, the transmission buffer remaining amount 108, and the information generation amount 109. The block to be determined and the specific region control unit 6 determine the final quantization step 114 based on the specific region information 113 notified from the outside and the quantization step 111 and the macroblock position information 112 calculated by the rate control unit 5. It is a block to do.

Here, an example of a hardware configuration for realizing each block of FIG. 1 is shown in FIG. For example, the camera unit 1 includes a lens 10 and an image signal processing circuit 12 having an image pickup device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor 11, a digital circuit, and a processor. The frame memory unit 2 includes a memory 13 such as an SDRAM (Synchronous Dynamic Random Access Memory). The image encoding unit 3 realizes image signal processing by an image encoding circuit 14 having a digital circuit, a processor, and the like. The transmission buffer unit 4 includes a memory 15 such as an SDRAM. The rate control unit 5 and the specific area control unit 6 have a configuration in which the processor 16 reads out and executes a program stored in the memory 17, and writes or reads data processed in the program to the memory 17. .

Next, the operation will be described.
In FIG. 1, the camera unit 1 converts a captured moving image into a digital signal and outputs an image signal 101. At this time, the camera unit 1 outputs the image signal 101 at the maximum processable frame rate.

The frame memory unit 2 rearranges the image signals 101 written in the shooting order in the encoding order according to the frame memory read control signal 102 and outputs the rearranged image signals 103 as the image signals 103. For example, FIG. 3 shows pictures to be written to and read from the frame memory unit 2 when M value (interval of I or P picture) = 3 and N value (interval of I picture) = 15.

The image encoding unit 3 performs intra-frame prediction or inter-frame prediction according to the picture type 110 and the quantization step 114 instructed by the rate control unit 5 so as to have a GOP (Group Of Pictures) structure as shown in FIG. Encoding using variable-length coding or the like is performed to output variable-length video stream data 105 and notify the rate control unit 5 of the information generation amount 109 of video stream data for each picture. The encoding method at this time is, for example, MPEG-2, H.264, or the like. H.264 / AVC (Advanced Video Coding), H.264. H.265 / HEVC (High-Efficiency-Video Coding), etc., and performs encoding processing in conformity with the standard.

The transmission buffer unit 4 temporarily stores the video stream data 104 in the memory, and outputs the video stream data 105 according to the read control signal 107 corresponding to the bit rate 106. Further, the remaining amount 108 of transmission buffer stored in the memory is calculated from the bit amount of the written video stream data 104 and the bit amount of the read video stream data 105.

For example, the rate control unit 5 is H.264. In the case of H.264 / AVC, the ITU-T H.264 The bit amount of the video stream data 104 is controlled so as to comply with a CPB (Coded Picture Buffer) buffer model defined by HRD (Hypothetical Reference Decoder) conformance described in the H.264 standard. For example, MPEG-5 TM5 (Test Model 5) is generally used as the bit amount control method, but the target information amount in GOP units is calculated from the bit rate 106, the transmission buffer remaining amount 108, and the information generation amount 109, The target information generation amount of the picture is calculated from the picture type 110 according to the GOP structure, and the quantization step 111 for each macroblock is controlled and output to the specific area control unit 6.

Here, the upper left position and lower right position of the rectangle are input as specific area information 113 to the specific area control unit 6 from the outside by user setting as position information of an important area to which particular attention is paid in the monitoring target area. The position information may be the upper left position of the rectangle, the horizontal size and the vertical size of the rectangle. There may be a plurality of specific areas. An example of the specific area information 113 is shown in FIG. First, the specific area control unit 6 calculates an area ratio of the specific area with respect to the entire screen at the timing of the picture cycle, and determines a reduction amount of the quantization step in the specific area according to the area of the specific area with respect to the entire screen. . When the area of the specific region is smaller than the predetermined threshold, the amount of decrease in the quantization step is increased, and when the area of the specific region is larger than the predetermined threshold, the amount of decrease in the quantization step is decreased. Similarly, the amount of increase in the quantization step for the outside of the specific region is determined according to the area of the specific region with respect to the entire screen. When the area of the specific region is smaller than the predetermined threshold, the increase amount of the quantization step is reduced, and when the area of the specific region is larger than the predetermined threshold, the increase amount of the quantization step is increased. At this time, a plurality of area thresholds may be provided, and the lowering and raising widths of the quantization step may be determined in stages.

Next, at the timing of the macroblock period, it is determined whether the own macroblock is inside or outside the specific area based on the specific area information 113 and the macroblock position information 112 notified from the rate control unit 5. Further, the lowering and lower limit values of the quantization step for the inside of the specific region and the increasing amount and upper limit value of the quantization step for the outside of the specific region are determined according to the size of the quantization step 111 in units of macroblocks. When the quantization step 111 is smaller than the predetermined threshold, the lowering or increasing width of the quantization step is increased, and when the quantizing step 111 is larger than the predetermined threshold, the lowering or increasing width of the quantization step is decreased. Further, an upper limit value and a lower limit value corresponding to the quantization step 111 are determined. Using the lowering and lower limit values of the quantization step inside the specific area determined as described above or the increase and upper limit values of the quantization step outside the specific area, addition / subtraction and limit processing are performed on the quantization step 111, and finally The quantizing step 114 is output to the image encoding unit 3. A flowchart of the quantization step control in the specific area control unit 6 is shown in FIG.

As described above, in the first embodiment of the present invention, when an important specific area is designated in the monitoring area, the quantization step is controlled according to the area of the area or the coding difficulty of the entire monitoring area. As a result, the image quality of a specific area can be improved without exceeding a predetermined bit rate.

Embodiment 2. FIG.
In the first embodiment described above, fluctuations in the amount of information generated per picture are suppressed by controlling the quantization steps inside and outside the specific region with respect to the quantization steps calculated by rate control in units of macroblocks. However, an embodiment in which the picture quality inside the specific area is improved by thinning out the picture without increasing the quantization step outside the specific area and lowering the frame rate will be described.

FIG. 6 is a block diagram showing the overall configuration of the image encoding apparatus in such a case. In the configuration, the specific area control unit 6 generates frame decimation information 115 based on the specific area information 113 and adds an output to the rate control unit 5 with respect to the first embodiment. The rate control unit 5 determines the GOP structure based on the frame decimation information 115 and adds the output of the frame decimation information 116 to the image encoding unit 3.

Next, the operation will be described while referring to differences from the first embodiment.
The specific area control unit 6 calculates an area ratio of the specific area with respect to the entire screen at the timing of the picture cycle, and determines a reduction amount of the quantization step in the specific area according to the area of the specific area with respect to the entire screen. The frame thinning information 115 is determined. When the area of the specific area is smaller than the predetermined threshold, the amount of decrease in the quantization step is increased and the rate of thinning out the frames is reduced (for example, 30 fps → 15 fps). In addition, the quantization step is reduced and the rate of thinning out frames is increased (for example, 30 fps → 5 fps). At this time, a plurality of area threshold values may be provided, and the reduction width of the quantization step and the ratio of thinning out the frames may be determined in stages. Further, the quantization step for the outside of the specific area is not changed (the increase width = 0).

The rate control unit 5 determines the GOP structure based on the frame decimation information 115 and outputs the picture type 110 and the frame decimation information 116 to the image encoding unit 3. The image encoding unit 3 determines the encoding target image based on the picture type 110 and the frame decimation information 116, and controls the frame memory read control signal 102 output to the frame memory unit 2 to realize frame decimation. . Also in this case, it is possible to obtain the same effect as in the first embodiment.

Embodiment 3 FIG.
In the first embodiment described above, fluctuations in the amount of information generated per picture are suppressed by controlling the quantization steps inside and outside the specific region with respect to the quantization steps calculated by rate control in units of macroblocks. However, without increasing the quantization step outside the specific area, the outside of the specific area in the case of a P picture or B picture is encoded in a coding mode (macroblock skip) that minimizes the amount of information generated. Thus, an embodiment in which the image quality inside a specific area is improved will be described.

FIG. 7 is a block diagram showing the overall configuration of the image encoding apparatus in such a case. In the configuration, the specific area control unit 6 generates specific area external information 117 based on the specific area information 113 and adds an output to the rate control unit 5 with respect to the first embodiment. The rate control unit 5 determines a coding mode for each macroblock based on the specific area external information 117 and the picture type 110, and adds the output of the coding mode information 118 to the image coding unit 3.

Next, the operation will be described while referring to differences from the first embodiment.
The specific area control unit 6 calculates an area ratio of the specific area with respect to the entire screen at the timing of the picture cycle, and reduces the quantization step reduction amount within the specific area and the specific area outside according to the area of the specific area with respect to the entire screen. Determine the coding mode information for. If the area of the specific area is smaller than the predetermined threshold, the quantization step is increased and the ratio of frames to be skipped by macroblocks is reduced (for example, 1/2 the number of P pictures in the GOP). If the area of the region is larger than a predetermined threshold, the quantization step is decreased and the ratio of frames for macroblock skip is increased (for example, 5/6 of the number of P pictures in the GOP). At this time, a plurality of area threshold values may be provided, and the quantization step reduction width and the ratio of frames to be macroblock skipped may be determined step by step. Further, the quantization step for the outside of the specific area is not changed (the increase width = 0).

Next, based on the specific area information 113 and the macro block position information 112 notified from the rate control unit 5 at the timing of the macro block period, it is determined whether the own macro block is inside or outside the specific area. In this case, the specific area external information 117 is output to the rate control unit 5. When the picture type is P picture or B picture, the rate control unit 5 determines the coding mode based on the specific area external information 117 and outputs the coding mode to the image coding unit 3. The image encoding unit 3 performs encoding according to the encoding mode information 118. Also in this case, it is possible to obtain the same effect as in the first embodiment.

Embodiment 4 FIG.
In Embodiments 1 to 3 described above, the quantization step is controlled according to the area of the specific area with respect to the entire screen. However, the quantization step is controlled according to the image characteristics of the specified specific area. An embodiment is shown.

FIG. 8 is a block diagram showing the overall configuration of the image encoding apparatus in such a case. In the configuration, output of the motion vector information 119, the intra macroblock information 120, and the variance value 121 of the luminance signal of the macroblock to the specific area control unit 6 is added from the image encoding unit 3 to the first embodiment.

Next, the operation will be described while referring to differences from the first embodiment.
The image encoding unit 3 has motion vector information 119 determined by motion prediction, intra (intraframe prediction) macroblock information 120, and a luminance signal variance value 121 in units of macroblocks calculated from an input image at the timing of the macroblock period. Is output to the specific area control unit 6. The specific area control unit 6 determines whether the own macroblock is inside or outside the specific area, and when it is inside the specific area, the motion vector information 119 or the intra macroblock information 120 or the variance value 121 of the luminance signal of the macroblock is used. Thus, the encoding difficulty level inside the specific area is calculated. It is difficult to encode a macroblock whose motion vector size in a specific area is larger than a predetermined threshold, a macroblock encoded as an intra macroblock, or a macroblock whose luminance signal has a variance value larger than a predetermined threshold. It is determined that the macro block has a high degree, and the number is counted. These macroblocks may be counted by any one condition or a combination of a plurality of conditions. After calculating the number of macroblocks having a high degree of difficulty in encoding within a specific area of one picture, the quantization for the inside of the specific area is performed in accordance with the proportion of macroblocks having a high degree of difficulty in encoding within the specific area at the timing of the next picture. Determine the step width.

When the number of macroblocks with high encoding difficulty is less than a predetermined threshold, the amount of decrease in the quantization step is increased, and when the number of macroblocks with high encoding difficulty is higher than a predetermined threshold, the quantization step Decrease the lowering range. Similarly, the amount of increase in the quantization step for the outside of the specific area is determined according to the ratio of macroblocks having a high degree of difficulty in encoding inside the specific area. If the number of macroblocks with high encoding difficulty is less than a predetermined threshold, increase the quantization step. If the number of macroblocks with high encoding difficulty is higher than the predetermined threshold, increase the quantization step. Increase Also in this case, it is possible to obtain the same effect as in the first embodiment.

Embodiment 5 FIG.
In the first to fourth embodiments described above, the specific area information 113 is notified from the outside. However, an embodiment in which an important area is detected based on the image of the shooting area to achieve high image quality will be described.

FIG. 9 is a block diagram showing the overall configuration of the image encoding apparatus in such a case. The configuration adds a specific area detection unit 7 to the first embodiment, and outputs a specific area flag 123 in units of macroblocks to the specific area control unit 6. Also, the motion vector information 119 and the macroblock color difference 122 are output from the image encoding unit 3 to the specific area detection unit 7.

Next, differences in operation from the first to fourth embodiments will be described.
The specific area detection unit 7 uses the motion vector information 119 input from the image encoding unit 3 to detect an area desired to have high image quality. The difference between the motion vector of each macroblock and the previous frame is calculated, and a moving average is calculated while holding values for a plurality of frames. When the moving average is larger than a predetermined threshold value, it is determined that a motion has occurred in the stationary area, and the specific area flag 123 for each macroblock is output. In addition, using the macroblock color difference 122 input from the image encoding unit 3, similarly, an area desired to have high image quality is detected. The skin color area is detected by comparing the color difference signals of the own macro block and the surrounding macro block with a predetermined threshold value. If the number of macro blocks in the continuous skin color area is smaller than the predetermined threshold value, the face area is determined. The unit specific area flag 123 is output. The specific area control unit 6 controls the quantization step in the same manner as in the first to fourth embodiments, and encodes the detected area as a specific area with high image quality.

In this case, the same effect as that of the first embodiment can be obtained.

As described above, in the present invention, when it is determined that the region is an important region according to the movement of the subject in the monitoring area and the size of the face, the quantization is performed according to the area of the region and the coding difficulty of the entire monitoring area. Since step control is performed, the image quality of an important area can be improved without exceeding a predetermined bit rate.

The image encoding apparatus according to the present invention can be applied as an image encoding apparatus that encodes a moving image or the like.

1 camera unit, 2 frame memory unit, 3 image encoding unit, 4 transmission buffer unit, 5 rate control unit, 6 specific region control unit, 7 specific region detection unit

Claims

A frame memory unit that rearranges input image signals from the input order to the encoding order, and an image code that encodes the image signals output from the frame memory unit using intra-frame prediction, inter-frame prediction, and variable-length encoding And
A transmission buffer unit that accumulates data encoded by the image encoding unit and transmits the data according to a predetermined bit rate;
Information on the specific area is input, and quantization steps inside and outside the specific area are controlled so that the encoded data can be within the predetermined bit rate, and the information on the quantization step is used as the image encoding unit. A specific area rate control unit to transmit to,
An image encoding apparatus comprising:
The specific area rate control unit controls quantization steps inside and outside the specific area according to a difficulty level of encoding the image signal at the predetermined bit rate. The image encoding device described.
The specific region rate control unit controls quantization steps inside and outside the specific region according to an area of the specific region in the image signal. The image encoding device described.
The specific region rate control unit controls a quantization step and a frame rate to be encoded in the specific region according to an area of the specific region in the image signal. The image encoding device according to claim 3.
2. The specific area rate control unit controls a quantization step and a macroblock coding mode inside the specific area according to an area of the specific area in the image signal. The image encoding device according to any one of claims 1 to 4.
The specific area rate control unit controls quantization steps inside and outside the specific area according to a magnitude of a motion vector of the specific area in the image signal. The image encoding device according to claim 5.
The specific area rate control unit controls quantization steps inside and outside the specific area according to the number of intra-frame prediction macroblocks in the specific area of the image signal. The image encoding device according to any one of claims 1 to 6.
The specific area rate control unit controls quantization steps inside and outside the specific area according to a magnitude of a variance value of a luminance signal of the specific area in the image signal. The image encoding device according to any one of claims 1 to 7.
A specific area detection unit that detects an important area according to the characteristics of the input image signal and outputs information on the area as information on the specific area to the specific area rate control unit;
The image encoding device according to claim 1, further comprising:
The specific area detecting unit detects the important area of the image signal according to a magnitude of a motion vector in units of macroblocks calculated from the image signal and a change in a time method. 9. The image encoding device according to 9.
The specific area detection unit detects the important area of the image signal according to the magnitude and distribution of a color difference signal for each predetermined size calculated from the image signal. The image encoding device according to claim 10.