KR20150002992A - Method for distributed vidio encoding and decoding of foreground block utilizing video analytics information - Google Patents

Abstract

An objective of the present invention is to provide a method for selectively encoding and decoding a foreground block by utilizing video analysis information, which is capable of reducing a calculation amount and improving the coding efficiency. To this end, a method of selectively encoding and decoding a foreground block by utilizing video analysis information according to the present invention includes the steps of: receiving a video frame divided into a key-frame and a wz-frame by an encoder; encoding the received video frame by the encoder; and decoding the encoded video frame received by a decoder. Thus, only the foreground block is selectively encoded by utilizing the video analysis information provided from an internal or external automatic video analysis module, so that the calculation amount may be reduced and the coding efficiency may be improved.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for selectively distributing video in a foreground block using video analysis information,

The present invention relates to a method for selectively encoding and decoding a foreground block using video analysis information. More particularly, the input video sequence for video security is classified into a static background of a wide area and a dynamic foreground area of a small size , Considering that the image quality of the image security application is mainly concentrated in the foreground area in the image security application, the foreground block information is extracted from the decoder, and the foreground block is selected by selecting the foreground block, so that the computation amount and bit resource consumed in the encoding of the encoder and the decoder And more particularly, to a method and apparatus for selectively encoding and decoding a foreground block using video analysis information.

Conventional video compression coding schemes consume compressed bit streams generated by a high performance and expensive single encoder in a plurality of low performance and low cost decoders. It is mainly suitable for media distribution channels such as terrestrial / cable / Internet broadcasting or CD / DVD / Blu-Ray.

Currently, the H.264 / AVC codec is applied to various channels, but the conventional video coding such as H.264 / AVC has a higher complexity than the decoder complexity. There is an existing problem.

In addition, Distributed Video Coding (DVC) is a coding technique suitable for a low-power terminal by allowing a decoder to perform calculations with high coding complexity in an encoder. However, the coding efficiency of existing distributed video coding techniques still does not meet the coding efficiency of H.264 / AVC.

Distributed video coding is not suitable for all existing video applications, but it can be applied to low-complexity resource-constrained applications such as wireless video sensor networks, wireless video security systems, and wireless nature ecological monitoring.

In most of these applications, the screen consists of a static background and a small number of foreground objects with a small or medium size. Also, the image quality of the foreground area is more important than the background area. Intra, Wyner-Ziv (WZ), and Skip have been proposed to introduce coding modes on a block-by-block basis to improve coding efficiency of a distributed video encoder.

However, in these techniques, since the encoding mode decision is performed by the encoder, an additional computation burden can be given to the encoder. In addition, the picture quality of side information (SI) generated by the decoder can not be reflected in the encoding mode decision. As a method of extending the coding mode decision, a technique has been proposed in which an encoder generates a background for a secure video sequence and extracts foreground blocks to selectively encode only foreground blocks.

However, since an operation such as background generation, background compression, and foreground extraction must be performed in the encoder, it is difficult to apply it to a distributed video encoder aiming at low-power encoding.

Korean Registered Patent No. 10-1059318 (Aug. 18, 2011)

Disclosure of Invention Technical Problem [8] The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide a Wyner-Ziv video coding structure and algorithm for a video security application, The present invention aims to provide a selective encoding and decoding method of a foreground block using video analysis information that selectively encodes a foreground block based on video analysis information provided by a video analytics module to reduce coding amount and improve coding efficiency .

According to another aspect of the present invention, there is provided a method for selectively encoding and decoding a foreground block using video analysis information, the method comprising: receiving a video frame in which an encoder is divided into a key frame and a WZ frame; Encoding a video frame received by the encoder; And a step of receiving and decoding the encoded video frame by a decoder.

The method for selectively encoding and decoding a foreground block using video analysis information according to the present invention selectively encodes only a foreground block based on video analysis information provided by an embedded or external automatic video analysis module, The coding efficiency can be improved.

1 is a diagram illustrating a transform domain Wyner-Ziv (TDWZ) codec structure to which a method of selectively encoding and decoding a foreground block using video analysis information according to the present invention is applied;
FIG. 2 is a diagram illustrating a pixel-domain Wyner-Ziv (PDWZ) codec structure to which a method of selectively encoding and decoding a foreground block using video analysis information is applied;
FIG. 3 is a conceptual diagram of an original frame, a foreground map, a background frame, and a subframe structure according to application of a selective encoding and decoding method of a foreground block using video analysis information according to the present invention,
4 is a flowchart illustrating a method of selectively encoding and decoding a foreground block using video analysis information according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms, and the inventor should appropriately interpret the concept of the term appropriately in order to describe its own invention in the best way. The present invention should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in this specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, It is to be understood that equivalents and modifications are possible.

1 is a block diagram of a transform domain Wyner-Ziv (transform-domain Wyner-Ziv; TDWZ) codec to which a method of selectively encoding and decoding a foreground block using video analysis information according to the present invention is applied. And a pixel-domain Wyner-Ziv (PDWZ) codec structure to which a selective encoding and decoding method of a foreground block using analysis information is applied.

As shown in FIG. 1 and FIG. 2, a codec to which a selective encoding and decoding method of a foreground block using video analysis information is applied includes an encoder 100 and a decoder 200.

In addition, referring to FIG. 3, a method of selectively encoding and decoding a foreground block using video analysis information according to the present invention will be described.

The encoder 100 performs a step of receiving a video frame (S100).

At this time, the encoder 100 separates the video frame into a key frame and a WZ frame, and receives the video frame.

The encoder 100 encodes the received video frame (S200).

If the Key-frame is received, the intra-picture coding module 110 such as H.264 Intra or JPEG embedded in the encoder 100 directly encodes the key-frame (S210).

Meanwhile, the video analysis module 210 built in the decoder 200 generates a background frame and a foreground map as shown in FIG.

In order to reduce the computation burden of the decoder 200, the background frame and the foreground map are generated in a video analysis module located outside the decoder 200, not in the embedded video analysis module 210, (Not shown).

That is, the video analysis module 210 may use an external video analysis module that is driven together with the decoder 200, or may incorporate a simple video analysis module in the decoder if it is difficult to use the external analysis module.

Both the built-in video analysis module and the external video analysis module must be able to generate a background frame and generate a foreground map and provide it to the proposed distributed video encoder.

Background frame and foreground map generation are operations that are common to most video analysis modules. The quality of the background frame may vary depending on the background generation technique, but does not affect the video coding structure or performance.

3 is a conceptual diagram of an original frame, a foreground map, a background frame, and a subframe structure according to application of a selective encoding and decoding method of a foreground block using video analysis information according to the present invention.

On the other hand, the background frame is updated as Equation (1) with the weighted average of the recently generated background frame and the recently decoded Key-frame. In addition, various background generation techniques can be used, and only a background frame can be provided to the proposed structure.

과 w는 n번째 복호된 key-프레임에서 화소 p의 값과 가중치를 나타낸다.From here

And w represent the value and weight of the pixel p in the n-th decoded key-frame.

The foreground map is calculated in units of M × M blocks as follows.

Where B and T _fg represent thresholds for block and foreground extraction. The foreground map is transmitted periodically by the encoder in consideration of the degree of change of foreground objects.

The generated foreground map is transmitted to the encoder 100.

When the encoder 100 receives the WZ-frame including the foreground map, the sub-frame construction module 120 built in the encoder 100 extracts only the blocks belonging to the foreground map for the WZ-frame One or more sub-frames are formed (S220).

As shown in FIG. 2, in the case of the PDWZ codec, the quantization module 140 of the encoder 100 performs a step of quantizing the pixels in the sub-frame configured in step S220 (S240).

The LDPC encoding module 150 of the encoder 100 separates and quantizes the quantized pixels into bit-planes in step S230 and performs encoding in step S250.

1, in the case of the TDWZ codec, the transcoding module 130 of the encoder 100 transforms and encodes the sub-frames constructed in step S220 into N × N block units (S230).

The quantization module 140 of the encoder 100 performs a step of forming a group for each of the coefficient positions in the transform block and quantizing the quantized result at step S240. The LDPC encoding module 150 of the encoder 100 quantizes And separates the coefficients into bit-planes and encodes them (S250).

The LDPC encoding module 150 of the encoder 100 multiplies the bit-plane by the parity generation matrix. If the foreground blocks can not fill all the sub-frames, the LDPC encoding module 150 of FIG. And " 0 "after the sub-frame.

More specifically, in most secure video sequences, the foreground area occupies a relatively small area within the frame. Thus, a sub-frame having a size of 1/4 of a half of the frame resolution is formed by using the foreground blocks.

The foreground blocks of the WZ-frame are filled in the sub-frame in the raster scan direction. If the number of foreground blocks is less than the number of blocks in the sub-frame, the non-filled blocks of the sub-frame are filled with a zero block as shown in FIG. 3 (d), as shown in FIG. If the foreground block remains after filling the first sub-frame, fill the next sub-frame and repeat this process if there are still foreground blocks remaining.

Group-of-pictures (GOPs) are defined as a set of one key frame and consecutive WZ-frames, and the GOP size is set to 2 or more. In the present invention, GOP size 2 is used as a reference. In this case, a frame index of a key frame is an even number, and a frame index of a WZ frame is expressed by an odd number.

After the encoding step (S200) of the video frame by the encoder (100), the decoder (100) decodes the frame (S300).

More specifically, the decoder 200 decodes the supplementary information blocks for the foreground blocks using the motion compensation interpolation (MCI) technique between the current key frame and the key frame immediately after the current WZ-frame, , And generates auxiliary information for the sub-frame using the generated auxiliary information block (S310).

That is, the auxiliary information for the sub-frame in the decoder 200 can be configured by generating auxiliary information for the foreground blocks in the foreground map.

The supplementary information for foreground blocks is generated through motion compensated interpolation (MCI). The MCI block for the foreground block B is interpolated using the motion vector v _B found in the motion search process performed between the immediately preceding key frame and the immediately following key frame.

The following is an example of using bi-directional motion estimation, and the interpolation method may vary depending on the motion estimation method.

For the motion estimation between the immediately preceding and following key frames, various conventional methods can be used. Forward motion estimation based on the block B of the immediately preceding key frame, backward motion estimation based on the key frame block B immediately after Directional and backward motion estimation, bi-directional motion estimation based on block B of WZ-frame, and deformed shape.

In the case of the TDWZ codec, the LDPC decoding module 220 of the decoder 200 performs LDPC decoding using the auxiliary information for the sub-frame generated in step S310 and the parity bits received from the encoder 210 (S320), an inverse quantization step by the inverse quantization module 230 is performed (S330), an inverse transformation step by the inverse transform decoding module 240 is performed (S340), and a decoded sub-frame is generated (S350).

In the case of the PDWZ codec, the decoder 200 performs decoding through the same procedure as in the case of the TDWZ codec, but the inverse conversion by the inverse transform decoding module 240 is omitted.

The frame synthesis module 250 included in the decoder 200 synthesizes the decoded sub-frame and the background frame generated after the step 'S330' or 'S340' into a final decoded frame using the foreground map (S360).

Finally, after the step 'S360', the video analysis module 210 of the decoder 200 generates and transmits a foreground map (S370).

After decoding the sub-frame using the sub information of the sub-frame and the received parity bits, the decoder 200 uses the background frame and the decoded sub-frame based on the foreground map by the frame synthesis module 250 And the final decoded frame is synthesized as follows.

는 보조정보 블록

과 패리티 블록을 이용하여 복호된 전경 블록의 화소 p의 값을 나타낸다.From here

≪ / RTI >

And the value of the pixel p of the decoded foreground block using the parity block.

The number of foreground blocks is related to the threshold value for extracting the foreground map of Equation (2). The larger the threshold value is, the smaller the number of blocks is determined as the foreground, the less energy is consumed for coding and transmission, and the decoded picture quality is degraded. The smaller the threshold value, the larger the number of blocks is determined as the foreground, and the energy consumption for coding and transmission is increased, but the decoding quality is improved.

This means that the inventive distributed video coding structure is scalable in terms of energy, bandwidth, and image quality, and they can be controlled using a threshold value for foreground extraction and a quantization parameter (QP). The threshold value is set to an average distortion value of key-frames encoded with the same quantization coefficient as that of the WZ-frame in general, so that the distortion of the background blocks can be controlled below a certain level.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It is to be understood that various modifications and changes may be made without departing from the scope of the appended claims.

100: encoder
110: encoding module
120: Sub-frame configuration module
130: Transcoding module
140: Quantization module
150: LDPC encoding module
200: decoder
210: Video analysis module
220: LDPC decoding module
230: Inverse quantization module
240: Inverse transform decoding module
250: frame synthesis module

Claims (10)

(a) receiving a video frame in which the encoder 100 is divided into a key frame and a WZ-frame;
(b) encoding the video frame received by the encoder (100); And
(c) receiving and decoding the encoded video frame by the decoder (200). The method of claim 1, further comprising:
The method according to claim 1,
In the step (b)
And encoding the key-frame by the encoding module (110) included in the encoder (100) if the key-frame is received in the encoder (100) A method for selectively encoding and decoding a foreground block utilized.
The method according to claim 1,
In the step (b)
When a background frame and a foreground map are generated in the video analysis module 210 or an external video analysis module built in the decoder 200 and the WZ-frame including the foreground map is transmitted to the encoder,
(b-1) extracting only blocks belonging to the foreground map for the WZ-frame by the sub-frame configuration module 120 of the encoder 100 to construct one or a plurality of sub-frames;
(b-3) In case of the PDWZ codec, the quantization module 140 of the encoder 100 quantizes the pixels in the sub-frame configured in the step (b-1);
(b-4) The LDPC encoding module 150 of the encoder 100 separates quantized pixels into bit-planes in step (b-3) and encodes them. A method for selectively encoding and decoding a foreground block using video analysis information.
The method of claim 3,
For the TDWZ codec,
Between the step (b-1) and the step (b-3)
(b-2) the transform coding module 130 of the encoder 100 transform-encodes the sub-primes constructed in step (b-1) into NxN block units,
In step (b-3), the quantization module 140 of the coder 100 forms a group for each coefficient position in the NxN transformed block, quantizes the group,
In the step (b-4), the LDPC encoding module 150 encodes the quantized coefficients into bit-planes and encodes the quantized coefficients into bit-planes, thereby selectively encoding and decoding the foreground block using the video analysis information. Decoding method.
The method according to claim 3 or 4,
The background frame is a weighted average of a background frame generated later and a key frame decoded later, and is updated by the following equation,

In the above equation

And w are values and weights of the pixel p in the n-th decoded key-frame.
The method according to claim 3 or 4,
The foreground map is calculated in units of M × M blocks by the following equation,

_Wherein B and T _fg are threshold values for block and foreground extraction, respectively. 2. The method of claim 1, wherein B and T _fg are threshold values for block and foreground extraction.
The method according to claim 3 or 4,
The step (c)
(c-1) The decoder 200 decodes the auxiliary information for the foreground blocks using a motion-compensation interpolation (MCI) technique between the current key frame and the key frame immediately following the current WZ- Generating a block and generating auxiliary information for the sub-frame using the generated auxiliary information block;
(c-2) In the case of the PDWM codec, the LDPC decoding module 220 of the decoder 200 extracts supplementary information on the sub-frame generated in the step '(c-1) LDPC decoding the pixels in the subframe encoded using the received parity bits; And
(c-3) generating (or constructing) a decoded sub-frame by dequantizing the pixels in the decoded sub-frame by the dequantization module 230 of the decoder 200 Selective coding and decoding method of foreground block using video analysis information.
8. The method of claim 7,
In the case of the TDWZ codec,
In step (c-3), the inverse transform step of the inverse transform decoding module 240 of the decoder 200 is further performed to form a decoded sub-frame. / RTI >
9. The method of claim 8,
After the step (c-3), the frame synthesis module 250 included in the decoder 200 decodes the decoded sub-frame generated after the step (c-3) and the decoded sub- To a final decoded frame using the foreground map and then generating and transmitting a foreground map by the video analysis module 210 of the decoder 200. [ A method for selectively encoding and decoding a foreground block. 10. The method of claim 9,
The final protection frame is synthesized by the following equation,

In the above equation

≪ / RTI >

And a pixel p of the decoded foreground block using the parity block. The method for selectively encoding and decoding a foreground block using video analysis information.

Images