WO2011084037A2

WO2011084037A2 - Distributed video coding/decoding method, distributed video coding/decoding apparatus, and transcoding apparatus

Info

Publication number: WO2011084037A2
Application number: PCT/KR2011/000194
Authority: WO
Inventors: 전병우; 김도형; 서덕영; 김철근; 심동규; 민경연; 박시내
Original assignee: 성균관대학교산학협력단
Priority date: 2010-01-11
Filing date: 2011-01-11
Publication date: 2011-07-14
Also published as: WO2011084037A3

Abstract

Disclosed are a distributed video coding/decoding method and a distributed video coding/decoding apparatus, which can improve loss resilience and the quality of service. The distributed video coding method first involves checking the state of a channel, determining a channel coding rate and the size of video data to be transmitted based on the checked state of a channel, determining the number of motion prediction performance steps based on the determined size of video data to be transmitted, coding the video data to be transmitted by performing motion predictions according to the determined number of motion prediction performance steps, and channel-coding the coded video data according to the determined channel coding rate. Accordingly, it is possible to improve loss resilience, even without additionally occupying network resources, thereby being capable of reducing the probability of decoding failure.

Description

Distributed video encoding / decoding method, distributed video encoding / decoding device, and transform encoding device

The present invention relates to image encoding and decoding, and more particularly, to a distributed video encoding / decoding method, a distributed video encoding / decoding apparatus, and a transform encoding apparatus that can be applied to a distributed video coding technique.

MPEG, H.26x, and other compression standards are widely used as efficient compression technologies for video players, customized video information services (VOD), video telephony, digital multimedia broadcasting (DMB), and video transmission in wireless mobile environments. The compression standards have a large gain in coding efficiency by eliminating temporal redundancy. As a representative method for reducing the temporal redundancy, there are motion prediction and compensation techniques. However, since the motion prediction and compensation technique requires a relatively large amount of computation in the video encoder, power consumption increases. Therefore, in a limited resource environment such as a sensor network, in order to reduce the power of the encoder, reducing the complexity of the encoder has emerged as an important technical problem.

Distributed video coding (DVC) technology based on the Slepian-Wolf theory and the Wyner-Ziv theory has attracted attention as a method of drastically reducing the complexity of the video encoding apparatus. The Slepian-Wolf theory mathematically proves that even if the correlated sources are encoded independently, the decoding gains can be equally obtained by performing the prediction encoding on each source together. The Wyner-Ziv theory extends the Slepian-Wolf theory, which is a lossless compression, to lossy compression. These two theories theoretically suggest the possibility of moving motion prediction and compensation, which eliminates inter-frame time redundancy, an important element of video coding, to the decoding device without loss of coding gain.

Wyner-Ziv coding technology is a representative method of distributed video coding technology, and generates side information of a current frame by using similarity between neighboring frames reconstructed by a decoding apparatus, and a difference between the generated auxiliary information and the current frame. Is regarded as virtual channel noise, and the encoding apparatus receives the parity bits generated by channel encoding to remove the noise included in the auxiliary information to restore the current frame.

In other words, the distributed video encoding technique reduces the complexity of the encoder by allowing the decoder to perform motion prediction, which takes up the largest amount of computation in the encoder. The encoder encodes video frames independently of each other, Since the video frames are not scanned to detect similarity, the amount of computation of the encoder can be reduced.

In the distributed video encoder and decoder to which the above-described distributed video encoding technique is applied, one of the most important parts in terms of coding efficiency is the prediction accuracy of the auxiliary information corresponding to the WZ frame in the decoder and the error probability of channel estimation in parity channel decoding. to be.

That is, the information used to generate parity bits through channel encoding in the distributed video encoder is an original WZ frame, and the information used when performing channel decoding using parity bits transmitted from the distributed video decoder is predicted WZ. Since the information is auxiliary information corresponding to the frame, the higher the similarity between the original WZ frame and the auxiliary information, the more the amount of parity bits used for channel decoding can be reduced, thereby reducing the amount of computation required.

However, in the conventional distributed video encoding method, since the encoder does not know the prediction accuracy of the auxiliary information generated by the decoder, it is necessary to transmit the parity bits and the quantized data having a fixed size to the decoder. There is a limit to improvement.

1 is a diagram illustrating a configuration of an encoder 110 and a decoder 130 corresponding to the conventional Wyner-Ziv coding technique.

The encoder 110 according to the Wyner-Ziv coding technique classifies pictures of source video content into two types. One is a frame to be encoded by using a distributed video coding method (hereinafter referred to as a 'WZ frame'), and the other is a picture to be encoded by a conventional coding method (hereinafter referred to as a 'key frame') rather than a distributed video coding method. )to be.

The keyframes are encoded by the intraframe encoding method of, for example, H.264 / AVC in the keyframe encoder 114 and transmitted to the decoder 130. The keyframe decoder 133 of the decoder 130 corresponding to the encoder 110 according to the conventional Wyner-Ziv coding technique reconstructs the transmitted keyframes. The auxiliary information generating unit 134 generates side information corresponding to the WZ frame by using the key frame restored by the key frame decoding unit 133 and converts the auxiliary information to the channel code decoding unit 131. Output

The auxiliary information generator 134 assumes linear motion between key frames located before and after the current WZ frame, and generates side information corresponding to the WZ frame to be restored using interpolation. In some cases, the interpolation method may be used instead of the interpolation method, but since the noise in the auxiliary information generated by the interpolation method is smaller than the noise in the auxiliary information generated by the interpolation method, the interpolation method is used in most cases. Meanwhile, in order to encode the WZ frame, the quantization unit 112 of the encoder 110 performs quantization on the WZ frame and outputs the quantization value of the WZ frame to the block unitization unit 112. The block unit unit 111 classifies the quantized value of the input WZ frame into predetermined coding units. The channel code encoder 113 generates a parity bit for each coding unit by using the channel code.

The generated parity bits are temporarily stored in a parity buffer (not shown), and are sequentially transmitted when the decoder 130 requests parity through a feedback channel. The channel code decoder 131 of FIG. 1 receives the parity transmitted from the encoder 110 and estimates the quantized value. The image restorer 132 of FIG. 1 receives the quantized value estimated by the channel code decoder 131 and inversely quantizes the quantized value to restore the WZ frame.

If the decoding fails because the parity bits provided from the encoder 110 are not sufficient to ensure successful decoding, the decoder 131 requests the encoder 110 to transmit more parity bits through a feedback channel. This process is repeated until decryption succeeds.

The distributed video encoding method using the feedback channel has the advantage of accurately performing channel modeling due to the above-described structure, but has a disadvantage in that the quality of service decreases due to a large transmission delay according to the feedback. .

A distributed video coding method that does not use a feedback channel (eg, PRISM: Power-efficient, Robust high-compression, Syndrome-based Multimedia Coding) performs channel modeling through a classifier of an encoding device and thereby uses a bit rate. Control is performed and success of decryption is checked through a cyclic redundancy check (CRC).

In other words, in the distributed video coding method that does not use the feedback channel, feedback is not required because the coding apparatus performs channel modeling, and thus the transmission delay is low because of the channel modeling. However, accurate channel modeling is performed because the coding apparatus performs channel modeling. This has the disadvantage of being difficult.

In the distributed video coding method that does not use the feedback channel, when the motion prediction is performed to compensate for the above-mentioned disadvantages, the accuracy of the channel modeling is improved, but the complexity of the coding apparatus increases, thereby reducing the advantages of the distributed video coding method. If the motion compensation is not performed, the accuracy of channel modeling decreases, thereby reducing the compression efficiency.

In the distributed video encoding technique described above, when channel loss occurs during data transmission, decoding fails. In such a case, in the case of distributed video encoding using a feedback channel, an additional parity bit may be requested to an encoding device, and decoding may be successful. However, there is a problem in that delay caused by feedback and usage of network resources increase. Alternatively, in the case of a distributed video encoding method that does not use a feedback channel, quality deteriorates due to decoding failure.

Conventional methods for compensating channel loss can be largely divided into a method using channel coding and a method of retransmission. The channel encoding method is a method of additionally transmitting a Forward Error Correction (FEC) packet when a channel loss occurs, and the retransmission method is a method of retransmitting a packet having a loss. However, both the method using channel coding and the method using retransmission have to transmit a larger amount of data than the original data to be transmitted, which leads to an increase in the use of network resources, thereby limiting its use when network resources are limited.

The distributed video encoding method based on the Wyner-Ziv theory is a technique of reconstructing a current WZ frame by correcting noise added to auxiliary information generated by a decoder using parity. Therefore, the less noise added to the generated auxiliary information, the smaller the amount of parity required. Therefore, it is important to generate auxiliary information well without noise in order to have good performance in terms of rate distortion.

The conventional auxiliary information generating method estimates a motion vector using a fixed size block (eg, 8x8) existing in the reconstructed key picture and reconstructs the motion vector of the auxiliary information to be restored in consideration of the distance between frames. Obtained from motion vectors between keyframes. The decoding unit in the key frame indicated by the motion vector of the auxiliary information thus obtained is generated as auxiliary information.

2 illustrates a conventional generation of auxiliary information using an interpolation method. The process of obtaining the motion vector of the auxiliary information to be reconstructed from the motion vector obtained by using a fixed size block between the reconstructed keyframes. It can be seen that 1/2.

3 illustrates a conventional generation of auxiliary information using extrapolation. The process of obtaining the motion vector of the auxiliary information to be reconstructed from the motion vector obtained by using a fixed size block between the reconstructed keyframes. Considering the distance between the pictures, the motion vector of the auxiliary information is determined by It can be seen that the same.

As such, when the motion vector of the auxiliary information to be generated is obtained from the motion vector using a fixed size block existing in the reconstructed keyframe, the motion is complicated between frames or the motion does not linearly change or an object And when the background suddenly disappears or appears, incorrect auxiliary information can be generated. In particular, when the size of the fixed block used for motion prediction between key frames increases, the accuracy of the obtained motion vector is similar to the motion of an actual image in a simple region such as a background, but not in a part where a complex object exists.

In contrast, when the size of the fixed block used for motion prediction between key frames decreases, the number of pixels used for estimating the motion vector decreases, so that the motion vector obtained through the motion prediction between key frames is obtained from the actual image. Frequent occurrences of inconsistencies with the movements of For this reason, 8x8 is generally used for the block size used for motion prediction for generating auxiliary information.

As in the above case, since the auxiliary information generated by the motion prediction using the fixed block size has a lot of noise, the transmitted parity cannot sufficiently remove the noise.

A first object of the present invention is to provide a distributed video decoding apparatus using variable block motion prediction.

A second object of the present invention is to provide a distributed video decoding method using variable block motion prediction.

It is a third object of the present invention to provide a distributed video encoding / decoding method capable of improving encoding efficiency and reducing processing complexity.

It is a fourth object of the present invention to provide a distributed video encoding / decoding apparatus that performs the distributed video encoding / decoding method.

A fifth object of the present invention is to provide a transform encoder that encodes an image encoded by the distributed video encoding method so that it can be decoded by a general decoder.

A sixth object of the present invention is to provide a distributed video encoding / decoding method capable of improving loss robustness and quality of service.

A seventh object of the present invention is to provide a distributed video encoding / decoding apparatus capable of improving loss robustness and quality of service.

An eighth object of the present invention is to provide an image encoding method using a plurality of image photographing apparatuses.

A ninth object of the present invention is to provide an image encoding method using a plurality of image photographing apparatuses.

A distributed video decoding apparatus using variable block motion prediction according to an aspect of the present invention for achieving the first object of the present invention includes a keyframe decoder for reconstructing a keyframe transmitted from a keyframe encoding apparatus, and the reconstructed key. A motion vector generator for generating a motion vector by determining a size of a block for performing a motion prediction using a frame and outputting the generated motion vector, and supplementary information using the reconstructed key frame and the generated motion vector. An auxiliary information generator to generate, a channel code decoder to estimate a quantized value using the parity bits transmitted from the distributed video encoding apparatus and the auxiliary information, and the quantized value and the auxiliary information estimated by the channel decoder Copy the current frame to be distributed video decoding based on Image restoration may include a to. The motion vector generator determines a plurality of blocks having different block sizes for motion prediction of the auxiliary information, obtains motion vectors between the reconstructed key frames based on the determined blocks, and obtains the motion vectors based on a predetermined criterion. A motion vector may be selected among the motion vectors. The motion vector generator is a block size determiner that determines a plurality of block sizes for motion prediction using the reconstructed key frame, and finds an area that matches a first block of one key frame among the reconstructed key frames. The region of one of the reconstructed key frames is searched to find a second block that most closely matches the first block, and the motion vector is a difference value between the first block and the second block. A motion prediction execution unit for generating a plurality of block sizes, a motion vector storage unit for storing the motion vectors generated by the motion prediction execution unit, and a first block of the one key picture among the motion vectors; The similarity between the second blocks of the other key picture is measured to obtain a motion vector of a block size corresponding to the highest similarity. It may include vector selection unit configured to select motion. The motion vector selector includes a sum of absolute difference (SAD) between the first block and the second block, a mean absolute difference (MAD), and an average sum of SSDs. The degree of similarity may be measured using any one of (of: Square Difference). The block size for performing the motion prediction may be at least one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 block sizes. The size of the block for performing the motion prediction may be determined by determining a change in motion between the reconstructed key frames. The block size determiner may determine a block size for performing the motion prediction to at least one of the block sizes of 16x16, 16x8, and 8x16 when the change in motion between the reconstructed key frames is small. The block size determiner may determine a block size for performing the motion prediction with at least one of block sizes of 8x8, 8x4, 4x8, and 4x4 when the motion change between the reconstructed key frames is large.

A distributed video decoding method using variable block motion prediction according to an aspect of the present invention for achieving the second object of the present invention comprises the steps of: reconstructing a key frame transmitted from a key frame encoding apparatus; Generating a motion vector by determining a block size for performing the motion prediction, generating auxiliary information using the reconstructed key frame and the generated motion vector, and a parity bit transmitted from a distributed video encoding apparatus; Estimating a quantized value using the auxiliary information, and reconstructing a current frame to be distributed video decoding using the estimated quantized value and the auxiliary information. Generating a motion vector by determining a block size for performing motion prediction using the reconstructed key frame may include determining a plurality of blocks having different sizes for motion prediction of the auxiliary information, and determining the plurality of determined blocks. The method may include obtaining motion vectors between the reconstructed keyframes and selecting a motion vector among the obtained motion vectors based on a predetermined criterion. Obtaining motion vectors between the reconstructed keyframes based on the determined plurality of blocks comprises: selecting among the reconstructed keyframes to find a region that matches a first block of one of the reconstructed keyframes. Searching an area of another key frame to find a second block that most closely matches the first block, and generating a motion vector for each of the plurality of block sizes, the motion vector being a difference value between the first block and the second block; It may include the step. Selecting a motion vector among the obtained motion vectors according to the predetermined criterion may include a first block of one key frame of the reconstructed key frames and a second key of the other one of the reconstructed key frames. Measuring a similarity between blocks for each motion vector and selecting a motion vector between the first block and the second block having the highest similarity among the similarities of the first block and the second block measured for each motion vector; It may include the step. The degree of similarity may include the sum of absolute difference (SAD) between the first block and the second block, the mean absolute difference (MAD), and the sum of square difference (SSD). The sum of squares of the differences of each block). The block size for performing the motion prediction may be at least one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 block sizes. The size of the block for performing the motion prediction may be determined by determining a change in motion between the reconstructed key frames. The block size determiner may determine a size of a block for performing the motion prediction with at least one of block sizes of 16x16, 16x8, and 8x16 when the change in motion between the reconstructed key frames is small. The block size determiner may determine a size of a block for performing the motion prediction with at least one of block sizes of 8x8, 8x4, 4x8, and 4x4 when a change in motion between the reconstructed key frames is large.

A distributed video encoding method for encoding an image divided into a first frame and a second frame according to an aspect of the present invention for achieving the third object of the present invention includes encoding and encoding the first frame. Transmitting one frame, receiving a motion vector from a distributed video decoding apparatus, generating auxiliary information corresponding to the second frame based on the provided motion vector, the generated auxiliary information and the second frame Obtaining a prediction error of the auxiliary information based on the quantization of the second frame based on the obtained prediction error. The distributed video encoding method may include transforming the input second frame, generating the auxiliary information, transforming the generated auxiliary information, and after the second frame is quantized, The method may further include generating parity bits by performing channel encoding on the second frame. Obtaining the prediction error of the auxiliary information based on the generated auxiliary information and the second frame may obtain the prediction error based on the converted auxiliary information and the converted second frame. Obtaining a prediction error of the auxiliary information based on the generated auxiliary information and the second frame may be performed by comparing the converted auxiliary information and the converted second frame to determine a location of a block in which an error has occurred. The method may include determining an amount of an error in the transformed auxiliary information, and obtaining a crossover probability based on the determined amount of the error and a length of an encoding code. Quantizing the second frame based on the obtained prediction error includes determining a number of quantization bits based on the obtained prediction error and configuring each second frame based on the obtained prediction error. And determining whether to execute quantization of the block. Determining whether to execute quantization of each block constituting the second frame based on the obtained prediction error does not occur for each block constituting the second frame, and the cross probability is determined in advance. Blocks smaller than the set threshold may omit quantization, and blocks having an error or having a cross probability greater than or equal to the threshold may perform quantization.

A distributed video decoding method according to an aspect of the present invention for achieving the third object of the present invention includes decoding a first encoded frame and generating a motion vector based on the at least one first decoded frame. Transmitting the generated motion vector to an encoding device; generating auxiliary information corresponding to a second frame using the generated motion vector; using parity bits and the auxiliary information provided from the encoding device. And correcting the auxiliary information, and inversely quantizing the corrected auxiliary information to restore the second frame.

According to an aspect of the present invention, there is provided a distributed video encoding apparatus configured to encode an input image divided into a first frame and a second frame, the encoder configured to encode the first frame; A motion compensator for generating auxiliary information corresponding to the second frame based on the motion vector provided from the decoding apparatus and the encoded first frame, and the auxiliary information based on the generated auxiliary information and the input second frame. The apparatus may include a quantization control unit that obtains a prediction error, quantizes the second frame based on the obtained prediction error, and a channel encoder that generates parity bits by channel coding the quantized second frame. The distributed video encoding apparatus may further include a first transformer configured to transform the generated auxiliary information and a second transformer configured to transform the input second frame. The quantization controller may obtain the prediction error based on the transformed auxiliary information and the transformed second frame. The quantization control unit compares the transformed auxiliary information with the transformed second frame to determine a location of a block in which an error occurs, determines an amount of an error in the converted auxiliary information, and determines the amount of the error. And by obtaining a crossover probability based on a length of an encoding code, the prediction error may be obtained. The quantization controller may determine the number of quantization bits based on the obtained prediction error and determine whether to perform quantization of each block constituting the second frame. The quantization control unit does not generate an error for each block constituting the second frame and omits quantization for a block whose cross probability is less than a preset threshold, and an error occurs or the cross probability is greater than or equal to the threshold. The block may perform quantization.

According to an aspect of the present invention, there is provided a distributed video decoding apparatus including: a decoder configured to decode an encoded first frame and a motion vector based on at least one decoded first frame. An auxiliary information generator for providing the generated motion vector to an encoding apparatus and generating auxiliary information corresponding to a second frame using the motion vector, a transformer for converting the generated auxiliary information, and converted auxiliary information And a channel decoder to correct the auxiliary information based on the parity bits provided from the encoding apparatus, and an inverse quantizer to dequantize the corrected auxiliary information to restore the second frame.

According to an aspect of the present invention, there is provided a transform encoding apparatus including: a decoder configured to decode an encoded first frame and a motion vector based on the decoded at least one first frame. An auxiliary information generator for providing the generated motion vector to an encoding apparatus and generating auxiliary information corresponding to a second frame using the motion vector, a transformer for converting the generated auxiliary information, converted auxiliary information, and A channel decoder to correct the auxiliary information based on the parity bits provided from the encoding apparatus, an inverse quantizer to dequantize the corrected auxiliary information to restore the second frame, and an encoder to encode the restored second frame can do. The encoder may encode the reconstructed second frame and the first frame decoded by the decoder. The encoder may include a first encoder that encodes the reconstructed second frame and a second encoder that encodes the first frame decoded by the decoder.

The encoding method of the distributed video encoding apparatus according to the aspect of the present invention for achieving the sixth object of the present invention described above comprises the steps of: identifying a channel condition, and determining a channel encoding rate and a size of video data to be transmitted based on the identified channel condition. Determining, based on the determined size of the video data to be transmitted, determining the number of motion prediction steps, encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps, and determining And channel encoding the encoded video data according to a channel coding rate. The checking of the channel condition may obtain an available bit rate and a packet loss rate (PLR). The determining of the channel coding rate and the size of the video data to be transmitted based on the identified channel situation may include determining the size of the video data to be transmitted and the size of the channel encoding data based on the available bit rate, the packet loss rate, and the channel coding rate. Can be. The determining of the number of motion prediction performing steps based on the determined size of the video data to be transmitted may determine the number of motion prediction performing steps among the total steps of TSS (Three Step Search) based on the determined size of the video data to be transmitted. . Encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps further includes providing the distributed video decoding apparatus with motion vector and motion prediction performance step information used for motion prediction. It may include.

In the encoding method of the distributed video encoding apparatus for achieving the sixth object of the present invention described above, the method may include: checking a channel condition, determining a channel coding rate and a size of video data to be transmitted based on the identified channel condition, Determining at least one method of adjusting the number of steps and adjusting the quantization interval based on the size of the video data as a method for adjusting the video data size, and encoding the video data to be transmitted according to the determined video data size adjusting method. And performing channel encoding on the encoded video data according to the determined channel coding rate. Determining at least one method of adjusting the number of motion prediction steps and adjusting the quantization interval based on the determined size of the video data to be transmitted is a method for adjusting the size of video data. At least one of the motion prediction performance step adjustment and the quantization interval adjustment may be determined as a method for adjusting video data size based on at least one prioritized priority among performances.

In the distributed video decoding method for achieving the sixth object of the present invention, the method includes reconstructing channel loss by channel decoding the provided encoded second data, and reconstructing the first frame by decoding the provided encoded first data. Generating auxiliary information by performing motion prediction on the reconstructed first frame based on the provided motion vector and information on the number of steps of performing motion prediction; correcting the auxiliary information generated using parity bits; And inversely quantizing the supplementary information to restore the second frame.

The distributed video encoding apparatus for achieving the seventh object of the present invention includes obtaining a first frame encoder that encodes an input first frame, a packet loss rate (PLR), and a channel coding rate based on the obtained packet loss rate. A channel monitor unit for determining a number of motion prediction steps based on the provided packet loss rate, and performing motion prediction according to the determined number of motion prediction steps and generating auxiliary information; And a second frame encoder configured to generate parity bits by encoding information and perform channel encoding according to the determined channel coding rate. The channel monitor may determine twice the packet loss rate as a forward error correction (FEC) rate. The second frame encoder is a transform and quantizer for transforming and quantizing the generated auxiliary information, a first channel encoder for generating parity bits by channel coding the transformed and quantized data, and a first channel according to the channel coding rate. A second channel encoder may perform channel encoding on the data provided from the encoder to compensate for channel loss. The motion prediction unit may provide the distributed video decoding apparatus with the motion vector and the motion prediction performance step information used for the motion prediction.

The distributed video encoding apparatus for achieving the seventh object of the present invention described above determines a channel encoding rate and a size of video data to be transmitted based on a first frame encoder encoding a first frame, a packet loss rate, and an available bit rate. A control unit for providing a control signal for adjusting video data size based on the determined size of the video data to be transmitted, a motion predictor for generating auxiliary information by performing motion prediction when a control signal is provided from the control unit; The second frame encoder may generate a parity bit by encoding the auxiliary information and perform channel encoding according to the determined channel coding rate. The controller determines at least one of the number of motion prediction steps and the quantization interval adjustment based on the determined size of video data to be transmitted as a method for adjusting video data size, and corresponds to a method for adjusting the determined video data size. A control signal can be provided. The motion prediction unit may perform motion prediction according to the number of motion prediction execution steps, which are control signals provided from the controller, and provide the distributed video decoding apparatus with motion vector and motion prediction execution step information used for the motion prediction. The second frame encoder is provided from a first channel encoder and a first channel encoder according to the first channel encoder according to the transform and quantization unit, the first channel encoder to generate a parity bit by channel encoding the transformed and quantized data And a second channel encoder configured to perform channel encoding on the data to compensate for channel loss. The transform and quantization unit may perform quantization according to the provided quantization interval information when quantization interval information is provided from the controller.

According to an aspect of the present invention, there is provided a method of encoding a plurality of distributed video encoding apparatuses, including: performing distributed video encoding on a photographed image using the plurality of distributed video encoding apparatuses; And transmitting the image data on which the distributed video encoding has been performed to a distributed video decoding unit through a predetermined communication channel, wherein distributed video encoding performed on at least one of the plurality of distributed video encoding apparatuses comprises: a first frame and A distributed video encoding method for encoding an input image divided into a second frame, the method comprising: encoding the first frame and transmitting an encoded first frame; receiving a motion vector from a distributed video decoding apparatus; Corresponding to the second frame based on a vector Generating pair information, obtaining a prediction error of the auxiliary information based on the generated auxiliary information and the second frame, and quantizing the second frame based on the obtained prediction error. Can be. The distributed video encoding method may include transforming the input second frame, generating the auxiliary information, transforming the generated auxiliary information, and after the second frame is quantized, The method may further include generating parity bits by performing channel encoding on the second frame. Obtaining the prediction error of the auxiliary information based on the generated auxiliary information and the second frame may obtain the prediction error based on the converted auxiliary information and the converted second frame. In the plurality of distributed video encoding apparatuses, at least one of the plurality of distributed video encoding apparatuses may be used as a depth image photographing apparatus for obtaining depth information of a subject, and the plurality of distributed video encoding apparatuses except the depth image photographing apparatus may include the subject. 3D may be photographed at a plurality of viewpoints, and a 3D image of the subject may be obtained based on the depth information provided from the depth image photographing apparatus and image information associated with the subject photographed at the plurality of viewpoints. The plurality of distributed video encoding apparatuses may include a predetermined sensor for detecting whether a fire occurs in at least one of the plurality of distributed video encoding apparatuses, and the image data photographed by the plurality of distributed video encoding apparatuses and whether or not the fire is provided by the sensor. The determination data can be transmitted using a wireless network. The plurality of distributed video encoding apparatuses photograph individual images according to a plurality of photographing viewpoints, and the photographed individual images are decoded by a distributed video decoding method and provided as a multiview screen having a plurality of screens, or by a specific control signal. Only some of the captured images may be provided. In the plurality of distributed video encoding apparatuses, when a monitored person appears in a surveillance region, motion is detected by motion sensors mounted in the plurality of distributed video encoding apparatuses, and the distributed video encoding apparatuses start to operate. After the distributed video encoding is applied to the encoding apparatus, the compressed data can be transmitted in the wireless sensor network network, and the compressed data is transmitted and received to the router through the wireless sensor network network by the wireless communication module included in the plurality of distributed video encoding apparatuses. In addition, the image data transmitted from the router to the gateway and transmitted to the gateway may be transmitted to the server through an Ethernet network. The plurality of distributed video encoding apparatuses may acquire an infrared image of a predetermined subject by using at least one of the plurality of distributed video encoding apparatuses as an infrared image photographing apparatus, and may include an image captured by at least one of the plurality of distributed video encoding apparatuses. The image quality of the infrared image may be compared to detect whether the subject is forged.

According to an aspect of the present invention, there is provided an image encoding method using a plurality of image capturing apparatuses. The image encoding method is distributed with respect to an image photographed using a distributed video encoder included in the plurality of image capturing apparatuses. Performing video encoding and transmitting the video data subjected to distributed video encoding to a distributed video decoding unit through a predetermined communication channel, wherein the distributed video encoding apparatus is performed in at least one of the plurality of distributed video encoding apparatuses. The video encoding may include determining a channel condition, determining a channel coding rate and a size of video data to be transmitted based on the identified channel condition, adjusting the number of motion prediction steps based on the determined size of the video data to be transmitted, and quantization interval Adjust at least one method of video data The method may include determining a method for controlling the data, performing encoding of video data to be transmitted according to the determined video data size adjusting method, and performing channel encoding on the encoded video data according to the determined channel coding rate. have. The checking of the channel condition may obtain an available bit rate and a packet loss rate (PLR). The determining of the number of motion prediction performing steps based on the determined size of the video data to be transmitted may determine the number of motion prediction performing steps among the total steps of TSS (Three Step Search) based on the determined size of the video data to be transmitted. . Encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps further includes providing the distributed video decoding apparatus with motion vector and motion prediction performance step information used for motion prediction. It may include. The plurality of distributed video encoding apparatuses may include at least one of the plurality of distributed video encoding apparatuses as a depth image capturing apparatus for obtaining depth information of a subject, and the plurality of distributed video encoding apparatuses except the depth image capturing apparatus may detect the subject. A 3D image of the subject may be obtained by photographing from a plurality of viewpoints, based on the depth information provided from the depth image capturing apparatus and image information related to the subject photographed from the plurality of viewpoints. The plurality of distributed video encoding apparatuses may include a predetermined sensor for detecting whether a fire occurs in at least one of the plurality of distributed video encoding apparatuses, and the image data photographed by the plurality of distributed video encoding apparatuses and whether or not the fire is provided by the sensor. The determination data can be transmitted using a wireless network. The plurality of distributed video encoding apparatuses photograph individual images according to a plurality of photographing viewpoints, and the photographed individual images are decoded by a distributed video decoding method and provided as a multiview screen having a plurality of screens, or by a specific control signal. Only some of the captured images may be provided. In the plurality of distributed video encoding apparatuses, when a monitored person appears in a surveillance region, motion is detected by motion sensors mounted in the plurality of distributed video encoding apparatuses, and the distributed video encoding apparatuses start to operate. After the distributed video encoding is applied to the encoding apparatus, the compressed data can be transmitted in the wireless sensor network network, and the compressed data is transmitted and received to the router through the wireless sensor network network by the wireless communication module included in the plurality of distributed video encoding apparatuses. In addition, the image data transmitted from the router to the gateway and transmitted to the gateway may be transmitted to the server through an Ethernet network. The plurality of distributed video encoding apparatuses may acquire an infrared image of a predetermined subject by using at least one of the plurality of distributed video encoding apparatuses as an infrared image photographing apparatus, and may include an image captured by at least one of the plurality of distributed video encoding apparatuses. The image quality of the infrared image may be compared to detect whether the subject is forged.

As described above, according to the distributed video encoding / decoding method, the distributed video encoding / decoding apparatus, and the transform encoding apparatus according to the embodiment of the present invention, motion prediction between key pictures when generating auxiliary information used in the decoding process of distributed video encoding is performed. The size of the block for variably is applied based on a predetermined criterion.

In addition, during distributed video encoding, a motion vector generated by a distributed video decoding apparatus is provided to a distributed video encoding apparatus, and the distributed video encoding apparatus generates auxiliary information using the motion vector, and then determines a prediction error of the generated auxiliary information. To control quantization.

In addition, the size of video data to be encoded is reduced by adjusting the complexity and / or quantization interval of the encoding apparatus according to channel conditions, and data is added and transmitted to compensate for channel loss by the reduced data amount.

Therefore, by more precisely estimating the motion vector between the key pictures, the performance of the reconstructed picture can be significantly improved. In other words, by estimating the motion vector while variably applying the size of the block existing in the reconstructed key picture, the motion vector of the auxiliary information to be generated can obtain a more sophisticated motion vector than the motion vector estimated using the conventional fixed size. The image quality of the reconstructed image can be improved by reducing the noise of the generated auxiliary information.

In addition, it is possible to improve encoding efficiency, obtain decoded images of high quality, and reduce computational complexity of the encoding apparatus and the decoding apparatus. In addition, a conventional encoding apparatus may decode an image encoded by the distributed video encoding method through the transform encoding apparatus to which the above-described distributed video encoding and distributed video decoding methods are applied.

In addition, the loss robustness can be improved without occupying additional network resources, thereby reducing the probability of decryption failure. In addition, since the use of the feedback channel can be minimized by reducing the probability of decoding failure, quality of service (QoS) of delay-sensitive services can be guaranteed.

1 is a block diagram illustrating a configuration of an encoder and a decoder corresponding thereto according to a conventional Weiner jib coding technique.

2 is a conceptual diagram illustrating a conventional generation of auxiliary information using an interpolation method.

3 is a conceptual diagram illustrating a conventional generation of auxiliary information using extrapolation.

4 is a block diagram illustrating a configuration of an apparatus for decoding a distributed video encoded image according to an embodiment of the present invention.

5 is a conceptual diagram illustrating a process of generating a motion vector according to an embodiment of the present invention.

6 is a block diagram illustrating a configuration of a motion vector generator according to an embodiment of the present invention.

7 is a flowchart illustrating a motion vector generation process according to an embodiment of the present invention.

8 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.

9 is a flowchart illustrating a prediction error determination step and a quantization step in more detail according to another embodiment of the present invention.

10 is a flowchart illustrating a distributed video decoding method according to another embodiment of the present invention.

11 is a flowchart illustrating a configuration of a distributed video encoding apparatus and a distributed video decoding apparatus according to another embodiment of the present invention.

12 is a block diagram showing a structure of a transform encoding apparatus according to another embodiment of the present invention.

13 is a block diagram showing the structure of a transform encoding apparatus according to another embodiment of the present invention.

14 is a block diagram illustrating a configuration of a transform encoder according to another embodiment of the present invention.

15 is a block diagram showing a configuration of a transform encoding apparatus according to another embodiment of the present invention.

FIG. 16 is a block diagram illustrating a configuration in the case where the transform encoding apparatus illustrated in FIG. 15 has scalability on the time axis.

17 is a graph illustrating a relationship between complexity and auxiliary information of an encoding apparatus.

18 is a graph illustrating a change in performance according to a motion prediction ratio of an encoding apparatus.

19 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.

20 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.

21 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.

22 is a block diagram illustrating a configuration of a distributed video encoding / decoding apparatus according to another embodiment of the present invention.

23 is a block diagram illustrating a configuration of a distributed video encoding / decoding apparatus according to another embodiment of the present invention.

24 is a conceptual diagram of a broadcast system using a plurality of video photographing apparatuses using distributed video coding according to an embodiment of the present invention.

25 is a conceptual diagram illustrating a media sharing system using a plurality of video photographing apparatuses using a distributed video encoding method according to an embodiment of the present invention.

FIG. 26 is a conceptual diagram of a 3D object tracking apparatus using a distributed video encoding method, according to an embodiment of the present invention.

27 is a block diagram illustrating a configuration of a multiview image processing apparatus according to an embodiment of the present invention.

28 is a conceptual diagram illustrating a surveillance camera system using distributed video encoding according to an embodiment of the present invention.

29 is a conceptual diagram of a monitoring system using a plurality of cameras according to an embodiment of the present invention.

30 illustrates a forest fire detection and ignition tracking system employing a wireless imaging device according to an embodiment of the present invention.

31 is a conceptual diagram illustrating a face detection system using a plurality of cameras according to an embodiment of the present invention.

32 is a conceptual diagram illustrating a distributed video encoding apparatus including a plurality of distributed video encoders according to an embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

The distributed video encoding and decoding apparatus according to an embodiment of the present invention includes a distributed video encoding apparatus 400 for encoding a wine-jib frame and a distributed video decoding apparatus including a function of generating auxiliary information through motion prediction using a variable block. Contains 430

Referring to FIG. 4, the distributed video encoding apparatus 400 includes a WZ frame encoder 410 and a keyframe encoder 420.

The WZ frame encoder 410 divides the frames of the source video content into wine-jib frames and key frames according to the wine-jib coding scheme. For example, even-numbered frames of source video content may be selected as key frames, or odd-numbered frames may be selected as wine-jib frames.

The WZ frame encoder 410 encodes the Weiner jib frames and the key frames and provides them to the distributed video decoding apparatus 430.

In addition, the distributed video decoding apparatus 430 according to the present invention includes a key frame decoder 460, a channel decoder 440, an image restorer 450, an auxiliary information generator 490, and a motion vector generator 480. ) And picture buffer 470.

The keyframe decoder 460 reconstructs the keyframe using the information received from the keyframe encoder 420.

The motion vector generator 480 determines the size of a block to be used for motion prediction using the reconstructed key frame.

The auxiliary information generator 490 generates auxiliary information for the current WZ frame to be reproduced by using the reconstructed key frame and the motion vector generated by the motion vector generator 480.

The channel decoder 440 estimates the quantized values by using the auxiliary information input from the auxiliary information generator 490 and the parity bits received from the distributed video encoding apparatus 400.

When the channel decoder 440 determines that a reliable quantized value cannot be estimated when performing channel code decoding, the channel decoder 440 continuously distributes parity bits within a predetermined limit until reliable estimation is possible. 400 may be configured to request and receive.

In this case, the distributed video encoding apparatus 400 receives only the parity amount necessary for decoding from the distributed video encoding apparatus 400, thereby being efficient in terms of rate-distortion performance. This is possible when there is a reverse channel (ie a feedback channel) requesting parity bits.

In order to alleviate this problem, the channel decoder 440 may be configured to receive a predetermined amount of parity bits at once without a parity request each time, so as not to request parity on the reverse channel while using the same. have.

Also, in the above case, after exhausting the parity bits sent, if the reliability is still low, the distributed video decoding apparatus 430 may further configure to request the parity bits to the distributed video encoding apparatus 400. . In addition, assuming that the reverse channel is not used, the distributed video encoding apparatus 400 sends a predetermined amount of parity calculated or set to the distributed video decoding apparatus 430, and the distributed video decoding apparatus 430 does not require parity bits. It may be configured in the form.

In addition, the channel code used in the channel decoder 440 of FIG. 4 may use a turbo code that is found to have reached a Shannon limit or may use an LDPC channel code. In addition, other channel codes with good coding efficiency and error correction can be used.

The image restorer 450 reconstructs the current WZ frame by using the quantized value and auxiliary information estimated by the channel decoder 440. The auxiliary information is generated by the auxiliary information generator 490.

The auxiliary information generator 490 generally uses an interpolation method assuming linear change between frames in generating auxiliary information. In this case, the change between video frames is caused by the movement of the object, the movement of the camera, the exposure area and the change of light. Except for changes in exposure area and light, the difference between these frames corresponds to the movement of pixels between frames. If the motion vector of the pixel is known correctly, it is possible to accurately predict the pixel of the frame to be interpolated.

That is, the auxiliary information generator 490 generates auxiliary information based on the motion vector provided from the motion vector generator 480.

The auxiliary information generator 490 uses side information corresponding to the WZ frame based on the motion vector provided from the motion vector generator 480 using the keyframe reconstructed by the keyframe decoder 460. Generates and outputs auxiliary information to the channel decoder 440. The auxiliary information generator 490 generates linear information corresponding to the WZ frame to be reconstructed using interpolation, assuming linear motion between key frames located before and after the current WZ frame.

The motion vector generator 480 generates a motion vector based on the key frames and provides the generated motion vector to the auxiliary information generator 490.

In detail, the motion vector generator 480 estimates the motion of a rectangular section or block of a frame.

Referring to FIG. 5, the motion vector generator 480 searches for another key frame 510 to find an MxN sample region 512 that matches the MxN sample block 502 of one key picture 500. Review area 520. The motion vector generator 480 compares all or part of the MxN block 502 of one frame with the possible MxN blocks in the search area (generally, the area based on the position of the current block) 520, and the most of them. Find the matching region 512. Subsequently, the motion vector generator 480 generates a motion vector that is a difference value between the position of the current M × N block 502 and the position of the candidate region 512.

Since the size of the macroblock for motion prediction varies in the size of the moving object in the video image, it may be more effective to use various block sizes for motion prediction. In the compression standard of H.264 / Advanced Video Coding (AVC), the block size for motion prediction may be one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4.

According to the present invention, the motion vector generator 480 determines sizes of a plurality of macroblocks for motion prediction, obtains motion vectors using macroblocks having respective sizes, and selects an appropriate motion vector among the motion vectors. do.

Referring to FIG. 6, the moved vector generator 480 according to the present invention includes a block size selector 600, a motion prediction performer 610, a motion vector storage 620, and a motion vector determiner 630. It includes.

The block size determiner 600 determines the size of the macroblock for motion prediction based on the key frame. The macroblocks may be square and / or rectangular and / or any shape in accordance with the present invention.

In other words, the block size determiner 600 determines a plurality of macroblock sizes. According to an embodiment of the present invention, the block size determining unit 600 may determine large block sizes for motion prediction when the motion is monotonous and there is no large difference between the key frames. If there is a lot of motion, small block sizes can be determined for motion prediction. Large block sizes include block sizes of 16x16, 16x8 and 8x16, and small block sizes may include 8x8, 8x4, 4x8 and 4x4 block sizes. According to another embodiment, the block size determiner 600 may determine the sizes of all macroblocks that can be used for motion prediction for motion prediction. When the block size determiner 600 determines the block sizes for motion prediction, the block size determiner 600 provides the determined block sizes to the motion prediction performer 610.

The motion prediction performer 610 generates motion vectors by performing motion prediction on each of sizes of blocks used for motion prediction. In detail, the motion prediction performing unit 610 searches an area of another key frame to find a sample area that matches the sample block of one key frame, finds the block that most matches the difference, and the difference therebetween. Create a motion vector as a value.

The motion prediction performer 610 repeats the process of generating and storing such a motion vector by the number of block sizes.

The motion prediction execution unit 610 stores the generated motion vector in the motion vector storage unit 620.

The motion vector selector 630 measures similarity between blocks by using motion vectors obtained with different block sizes present in a key frame, and selects a motion vector determined to be the most similar among them. That is, the motion vector selector 630 measures a similarity between the block of the one key frame and the corresponding block of the other key frame among the motion vectors and selects a motion vector having a block size corresponding to the highest similarity. Choose.

Here, the similarity between each block present in the key frame is generally the sum of absolute difference (SAD) or mean absolute difference (MAD) of the blocks present in each key frame. It can be calculated using a method such as the average of the absolute sum of) or SSD (Sum of Square Difference, sum of squares of the difference of each block). In addition, it is also possible to measure the similarity between each block existing in the key frame by applying other appropriate methods in addition to the above-described similarity measuring method.

The motion vector selector 630 provides the selected motion vector to the auxiliary information generator 490.

The operation of the motion vector generator 480 having such a configuration will be described below.

Referring to FIG. 7, the motion vector generator 480 first determines a plurality of macroblock sizes for motion prediction in step S710.

In this case, macroblock sizes may be determined according to characteristics of an image. For example, if there are fine or complex motions between frames or there is a lot of motion, small block sizes may be determined for motion prediction. Alternatively, when the motion between frames is monotonous and there is no big difference, large block sizes may be determined for motion prediction. Alternatively, the sizes of all macroblocks that can be used for motion prediction may be determined.

Subsequently, the motion vector generator 480 generates a motion vector by performing motion prediction according to the size of each macro block in step S720. Specifically, the area 520 of the other key frame 510 is searched to find the MxN sample area 512 that matches the MxN sample block 502 of one frame 500. The motion vector generator 480 compares all or part of the MxN block 502 of one picture with the possible MxN blocks in the search area (generally, the area based on the position of the current block) 520, and the most among them. The matching area, that is, the candidate area 512 is found. Subsequently, the motion vector generator 480 generates a motion vector that is a difference value between the position of the current M × N block 502 and the position of the candidate region 512.

Subsequently, the motion vector generator 480 determines whether a motion vector has been generated for the sizes of all macroblocks determined in operation S730. If a motion vector has been generated for all determined macroblock sizes, the motion vector generator 480 measures similarity between a block of one picture and a block of another frame for each motion vector in step S740. According to an embodiment of the present invention, the block may have a square shape. In addition, as a method of measuring similarity between blocks existing in a key frame, a sum of absolute difference (SAD), a mean absolute difference (MAD), or a sum of square difference (SSD) may be used. .

However, the scope of the present invention is not limited thereto, and the block size may be configured in any form other than square, and in the method of measuring similarity between blocks existing in a key frame, another arbitrary method may be selected. Of course. In addition, the present invention can be applied in any case, such as the case of performing the auxiliary information by the interpolation method, the extrapolation method as shown in FIG.

Subsequently, the motion vector generator 480 selects a motion vector corresponding to the highest similarity among the generated motion vectors. That is, a motion vector having the largest similarity between blocks can ensure the best motion compensation.

The motion vector generator 480 outputs the motion vector selected in step S760 to the auxiliary information generator 490.

As described above, in the decoding in the Wyner-Ziv coding technique according to the preferred embodiment of the present invention, a method and apparatus for using selective hash information can be made. Meanwhile, in the above description of the present invention, specific embodiments will be described. However, there can be various embodiments within the scope of various modifications do not depart from the gist of the present invention. Therefore, the scope of the present invention should not be defined by the described embodiments, but by the claims and equivalents of the claims.

Referring to FIG. 8, the distributed video encoding apparatus first encodes an input key frame and transmits the encoded key frame to the distributed video decoding apparatus (step S810). Here, the distributed video encoding apparatus may encode the key frame through various known encoding techniques such as H.264 / AVC and MPEGx.

Subsequently, the distributed video encoding apparatus receives a motion vector from the distributed video decoding apparatus (step S820), performs motion compensation on a key frame encoded using the received motion vector, and generates auxiliary information corresponding to the WZ frame. (Step S830). Since the generated auxiliary information is the same as the auxiliary information generated by the distributed video decoding apparatus, the distributed video encoding apparatus may know the auxiliary information generated by the distributed video decoding apparatus.

Then, the distributed video encoding apparatus transforms the generated auxiliary information (step S840).

In addition, the distributed video encoding apparatus converts the input original WZ frame (step S850), and determines a prediction error of the auxiliary information based on the converted original WZ frame and the auxiliary information converted in step S840 (step S860). Here, the conversion time point for the input original WZ frame is not limited to the order shown in FIG. 8, and may be executed corresponding to the time point at which the original WZ frame is input.

When the prediction error is determined, the distributed video encoding apparatus performs quantization based on the determined prediction error (step S870), and then performs parity bit generation by performing channel encoding on the quantized data to distribute the generated parity bits. Provided to the video decoding apparatus (step S880). Here, the distributed video encoding apparatus may perform quantization for each coding unit, that is, an image area having a predetermined size that is encoded together at one time, such as a macro block.

Referring to FIG. 9, in the distributed video encoding apparatus, in order to determine a prediction error of an auxiliary information frame, the amount of error (Count _coeff.block ) generated in the auxiliary information frame, the position of the block in which the error occurs (M _error ), and the crossing probability (Crossover) is calculated (step S860). The calculated values are used to determine the quantization level of each block and whether to perform quantization.

In detail, a position M _error in which an error occurs in the auxiliary information frame may be calculated using Equation 1 below.

Equation 1

In Equation 1, ORI means an original WZ frame, and SI means an auxiliary information frame. i is the index of the block and I is the number of blocks in the frame. j is the index of the coefficient in the block, and J is the number of coefficients in the block. In addition, k is an index of a bit plane, and when k is 0, it means the most significant bit, and as k increases, it means a lower bit. K means the number of bit planes in the coefficient.

In addition, the amount of errors in the auxiliary information frame may be calculated using Equation 2 below.

Equation 2

Since the size of the original WZ frame is larger than the LDPCA code (LDPCA) code length, encoding is performed by dividing the entire frame without encoding all at once. Crossover probability may be measured using Equation 3 using the amount of error and the LDPCA code length in the range of the original WZ frame that is performed at one time.

Equation 3

After calculating the position of the error, the amount of error, and the crossover

probability using Equations

1, 2, and 3 as described above, the distributed video encoding apparatus determines the quantization level of the original WZ frame based on the calculated values ( Step S900). For example, the distributed video encoding apparatus may determine the quantization level by determining the quantization bit based on the calculated values.

Subsequently, the distributed video encoding apparatus determines whether an error occurs and an intersection probability is smaller than a preset threshold for each block in the WZ frame (steps S910 and S920), so that an error does not occur and the intersection probability is increased. Quantization is omitted for blocks smaller than the threshold value (step S930), and quantization is performed for blocks in which an error has occurred or crossover probability is greater than or equal to the threshold value (step S940).

The distributed video encoding apparatus performs channel encoding when quantization of all blocks in a frame is completed through steps S910 to S940.

As described above, the distributed video encoding method according to an embodiment of the present invention adaptively performs quantization for a block determined to be an error-free block by using M _error indicating an error of a block. That is, if an error is concentrated in a block having an error, quantization is performed to lower the crossing probability, and if only the block having the error is encoded, if the crossing probability is low enough, the block without an error does not perform quantization. This allocates a large number of bits to an error-prone block, and allocates bits according to the crossing probabilities to improve the performance of the LDPCA, thereby enabling efficient use of the bits.

In the distributed video encoding method according to an embodiment of the present invention, assuming that the distributed video encoding apparatus knows the channel decoding capability of the distributed video decoding apparatus, it is possible to decode the ratio of bits without errors and bits with errors in the encoding apparatus. Transmit according to the number of crossing probabilities. Therefore, although the amount of bits to be transmitted may increase according to the image, the probability of decoding failure is reduced as much as possible, thereby increasing the safety of the decoder and effectively using the bits.

Referring to FIG. 10, first, a distributed video decoding apparatus decodes an encoded key frame transmitted from a distributed video encoder (step S1010). Here, the distributed video decoding apparatus may decode a key frame through various known decoding techniques such as H.264 / AVC and MPEGx.

Subsequently, the distributed video decoding apparatus generates a motion vector based on the decoded key frame (step S1020), and transmits the generated motion vector to the distributed video encoder (step S1030).

The distributed video decoding apparatus generates motion information by performing motion compensation using the generated motion vector (step S1040). For example, the distributed video decoding apparatus estimates a motion vector between reconstructed key frames and then obtains a motion vector of auxiliary information corresponding to a WZ frame to be reconstructed from the motion vectors between the reconstructed key frames in consideration of the distance between the frames. do. The decoding unit in the key frame indicated by the motion vector of the obtained auxiliary information may be generated as auxiliary information.

As described above, when the auxiliary information is generated, the distributed video decoding apparatus corrects the noise included in the generated auxiliary information by using the parity bits transmitted from the distributed video encoding apparatus (step S1050), and then performs dequantization to perform a WZ frame. To be restored (step S1060).

Referring to FIG. 11, the distributed video encoding apparatus 1100 may include a keyframe encoder 1105, a buffer 1110, a motion compensator 1115, a first transform unit 1120, a second transform unit 1125, It may include a quantization control unit 1130 and a channel encoder 1140.

First, an input image to be encoded is classified into a key frame and a WZ frame so that the key frame is provided to the key frame encoder 1105 and the WZ frame is provided to the second converter 1125. After the key frame encoder 1105 encodes the key frame, the key frame encoder 1105 provides the encoded key frame to the motion compensator 1115 and the key frame decoder 1155 of the distributed video decoding apparatus. Here, the key frame encoder 1105 may encode the key frame through various known encoding techniques such as H.264 / AVC and MPEGx.

The buffer 1110 temporarily stores a motion vector (MV: Motion Vector) provided from the auxiliary information generator 1160 of the distributed video decoding apparatus 1150 and provides the buffer to the motion compensator 1115.

The motion compensator 1115 reads a motion vector transmitted from the distributed video decoding apparatus 1150 and stored in the buffer 1110, and then performs motion compensation on a key frame encoded by using the read motion vector. After generating the information, the generated auxiliary information is provided to the first converter 1120. Here, since the motion compensator 1115 performs motion compensation on a key frame using the motion vector provided from the distributed video decoding apparatus 1150, the motion compensator 1115 generates the distributed video encoding apparatus 1100 and the distributed video decoding apparatus 1150. The supplementary information is identical to each other, and thus, the distributed video encoding apparatus 1100 may recognize the supplementary information generated by the distributed video decoding apparatus 1150 and reflect the supplementary information in the encoding.

The first transform unit 1120 transforms the auxiliary information provided from the motion compensator 1115 to provide the quantization control unit 1130, and the second transform unit 1125 transforms the input original WZ frame to quantize it. The control unit 1130 is provided. Here, the first and

second converters

1120 and 1125 may use the same conversion method.

The quantization controller 1130 determines the prediction error of the auxiliary information based on the converted auxiliary information frame provided from the first transform unit 1120 and the original WZ frame provided from the second transform unit 1125, and then based on the determined prediction error. The quantization level and quantization of each block constituting the original WZ frame are determined, and the quantization is performed based on the quantization level and then provided to the channel encoder 1140.

The channel encoder 1140 generates a parity bit for the quantized data using the channel code, and then converts the generated parity bit to the channel decoder 1170 of the distributed video decoding apparatus 1150. send. In this case, the generated parity bit may be stored in a separate buffer (not shown) and then provided to the distributed video decoding apparatus according to a transmission request of the distributed video decoding apparatus.

The distributed video decoding apparatus 1150 may include a keyframe decoder 1155, an auxiliary information generator 1160, a transformer 1165, a channel decoder 1170, and an inverse quantizer 1175.

The keyframe decoder 1155 decodes the encoded key frame provided from the keyframe encoder 1105 of the distributed video encoding apparatus 1100 and provides the decoded keyframe to the auxiliary information generator 1160. The key frame decoder 1155 may decode the key frame through various known decoding techniques such as H.264 / AVC and MPEGx.

The auxiliary information generator 1160 performs motion prediction using the decoded key frame and then performs motion compensation to generate auxiliary information corresponding to the WZ frame. In addition, the auxiliary information generator 1160 provides the motion vector generated in the process of performing the motion prediction to the distributed video encoding apparatus 1100. Here, the auxiliary information generator 1160 estimates a motion vector between the reconstructed key frames, and then calculates a motion vector of the auxiliary information corresponding to the WZ frame to be restored from the motion vectors between the reconstructed key frames in consideration of the distance between the frames. A decoding unit in a key frame indicated by the motion vector of the obtained auxiliary information may be generated as auxiliary information.

The converter 1165 converts the auxiliary information provided from the auxiliary information generator 1160 and provides the converted information to the channel decoder 1170, and the channel decoder 1170 is included in the converted data provided from the converter 1165. The quantized data is estimated by correcting the noise by using parity bits transmitted from the distributed video encoding apparatus 1100.

The inverse quantizer 1175 receives the estimated quantization data from the channel decoder 1170 and inversely quantizes the WZ frame. Here, the ambiguity generated during inverse quantization is solved by referring to the auxiliary information provided from the auxiliary information generator 1160.

As illustrated in FIG. 11, in the distributed video encoding apparatus and the distributed video decoding apparatus, the distributed video decoding apparatus 1150 uses the motion vector used to predict auxiliary information corresponding to the WZ frame. The auxiliary information is transmitted to the encoding apparatus 1100 and the distributed video encoding apparatus 1100 generates auxiliary information by performing motion compensation using the received motion vector, thereby providing the same auxiliary information as that generated by the distributed video decoding apparatus 1150. After obtaining, the coding efficiency can be improved by calculating a prediction error between the generated auxiliary information and the original WZ frame and adjusting the quantization level and whether the quantization is based on the prediction error, thereby obtaining an image of excellent quality.

FIG. 12 is a block diagram illustrating a structure of a transform encoding apparatus according to an embodiment of the present invention. For example, the distributed video encoding apparatus 1100 and the distributed video decoding apparatus 1150 illustrated in FIG. 11 are applied. Shown. The distributed video decoding apparatus 1200 and the distributed video decoding apparatus 1260 included in the transform encoding apparatus 1250 illustrated in FIG. 12 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.

The trans-coder may decode an image encoded by the distributed video encoding apparatus 1200 according to an embodiment of the present invention in a decoding apparatus 1295 that does not perform conventional distributed video decoding. Perform the function of converting.

Referring to FIG. 12, the transform encoding apparatus 1250 according to an embodiment of the present invention may include a distributed video decoding apparatus 1260 and an encoder 1290.

The distributed video decoding apparatus 1260 decodes an encoded key frame provided from the distributed video encoding apparatus 1200, generates auxiliary information using the decoded key frame, and distributes a motion vector generated in the auxiliary information generation process. The video encoding apparatus 1200 is provided.

In addition, the distributed video decoding apparatus 1260 corrects auxiliary information generated by using the parity bits provided from the distributed video encoding apparatus 1200, and then dequantizes and restores the WZ frame.

As described above, the key frame and the WZ frame reconstructed by the distributed video decoding apparatus 1260 are provided to the encoder 1290, and the encoder 1290 encodes the received key frame and the WZ frame and then decodes the decoder ( 1295). Here, the encoder 310 may encode using various known coding techniques such as H.264 / AVC, MPEGx, for example. Preferably, the same standard technique as the decoding standard technique applied to the decoding apparatus is used. It can be applied to the encoding.

The distributed video decoding apparatus 1300 illustrated in FIG. 13 and the distributed video decoding apparatus 1360 included in the transform encoding apparatus 1350 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.

Referring to FIG. 13, the transform encoding apparatus 1350 according to another embodiment of the present invention may include a distributed video decoding apparatus 1360, a first encoder 1390, and a second encoder 1393.

The distributed video decoding apparatus 1360 decodes an encoded key frame provided from the distributed video encoding apparatus 1300, generates auxiliary information using the decoded key frame, and distributes a motion vector generated in the auxiliary information generation process. The video encoding apparatus 1300 is provided.

In addition, the distributed video decoding apparatus 1360 corrects auxiliary information generated by using the parity bits provided from the distributed video encoding apparatus 1300, and then dequantizes and restores the WZ frame.

As described above, the WZ frame reconstructed by the distributed video decoding apparatus 1360 is provided to the first encoder 1390, and the reconstructed key frame is provided to the second encoder 1393. The first encoder 1390 encodes the reconstructed WZ frame and provides it to the first decoding device 1396, and the second encoder 1393 encodes the reconstructed key frame and provides the second decoding device 1399. do. Here, the first encoder 1390 and the second encoder 1393 may encode through various well-known encoding techniques such as H.264 / AVC, MPEGx, and not the distributed video encoding method. The same standard technique as that applied to the corresponding decoding apparatus for providing the corrected data may be applied to the encoding.

14 is a block diagram illustrating a structure of a transform encoder according to another embodiment of the present invention, in which a transform encoder 1450 to which the distributed video encoding apparatus 1400 and the distributed video decoding apparatus 1460 are applied is scalable in a time axis. For example, the case having scalablity is illustrated. The distributed video decoding apparatus 1400 illustrated in FIG. 14 and the distributed video decoding apparatus 1460 included in the transform encoding apparatus 1450 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.

Referring to FIG. 14, the transform encoding apparatus 1450 may largely include a base layer transform encoding unit 1455 and an enhancement layer transform encoding unit 1457. The base layer transform encoder 1455 may include a decoder 1465 and a second encoder 1493, and the enhancement layer transform encoder 1457 may include an auxiliary information generator 1470 and a converter 1475. , A channel decoder 1480, an inverse quantizer 1485, and a first encoder 1490.

The base layer transform encoder 1455 decodes an encoded key frame provided from the distributed video encoding apparatus 1400, and then re-codes the encoded key frame through the second encoder 1493 to provide it to the second decoding apparatus 1499. In addition, the base layer transform encoder 1455 provides the decrypted key frame to the enhancement layer transform encoder 1457.

The enhancement layer transform encoder 1157 generates auxiliary information based on the key frame provided from the base layer transform encoder 1455, and corrects the generated auxiliary information using the parity bits provided from the distributed video encoding apparatus 1400. After inverse quantization, the signal is encoded by the first encoder and then provided to the first decoder 1496. In this case, the enhancement layer transform encoder 1457 provides the distributed video encoding apparatus 1400 with the motion vector generated during the auxiliary information generation process. The first encoder 1490 and the second encoder 1493 may encode through various known encoding techniques such as H.264 / AVC, MPEGx, and the like, rather than distributed video encoding. The same standard technology as that applied to the corresponding decoding apparatus for providing the same may be applied to the encoding.

As shown in FIG. 14, the transform encoding apparatus according to another embodiment of the present invention encodes a key frame reconstructed by the base layer transform encoder 1455 and reconstructed by the enhancement layer transform encoder 1575. The WZ frame is encoded and then provided to the second decoding unit 1499 and the first decoding unit 1496, respectively, so that the base layer outputs only the result decoded by the second decoding unit 1499, and the enhancement layer. Next, the resultant images decoded by the second decoding unit 1499 and the first decoding unit 1496 may be rearranged and output.

The distributed video encoding apparatus 1500 and the distributed video decoding apparatus 1560 included in the transform encoding apparatus 1550 illustrated in FIG. 15 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.

Referring to FIG. 15, a key frame encoded by the distributed video encoding apparatus 1500 is provided to the key frame decoder 1565 of the transform encoding apparatus 1550 and simultaneously provided to the second decoding apparatus 1599. That is, the key frame decoded by the decoding unit 1565 of the transcoding apparatus 1550 is provided only to the auxiliary information generating unit 1570 to be used only for generating auxiliary information, and provides the decoded key frame to an external decoding device. I never do that. Therefore, the image decoded by the second decoding apparatus 1599 is an image encoded by the keyframe encoder 1505 of the distributed video encoding apparatus 1500, and the transform encoding apparatus 1550 decodes the encoded key frame by second decoding. No separate processing to provide to the device 1599 is performed.

In addition, the transform encoding apparatus 1550 corrects and inverse quantizes auxiliary information generated by using the parity bits provided from the distributed video encoding apparatus 1500, and then encodes the encoded information through the first encoder 1593 and then encodes the first decoding apparatus. Provided in (1596).

Referring to FIG. 16, the transform encoding apparatus 1650 may largely include a base layer transform encoder 1655 and an enhancement layer transform encoder 1657. The base layer transform encoder 1655 may include a decoder 1665. The enhancement layer transform encoder 1657 may include an auxiliary information generator 1670, a converter 1675, a channel decoder 1680, and the like. The dequantizer 1685 and the first encoder 1690 may be included.

The base layer transform encoder 1655 decodes the encoded key frame provided from the distributed video encoding apparatus 1600 and provides the decoded key frame to the auxiliary information generator 1670. In this case, the key frame decrypted by the base layer transform encoder 1655 is provided only to the auxiliary information generator 1670 of the enhancement layer transform encoder 1657 and used only to generate the auxiliary information. It is not provided to an external decoding device. Accordingly, the image decoded by the second decoding apparatus 1696 is an image encoded by the encoder 1605 of the distributed video encoding apparatus 1600, and the base layer transform encoder 1655 decodes the encoded key frame by second decoding. There is no separate processing to provide to the device 1696.

The enhancement layer transform encoder 1657 generates auxiliary information based on the key frame provided from the base layer transform encoder 1655, and corrects the generated auxiliary information using the parity bits provided from the distributed video encoding apparatus 1600. After inverse quantization, the signal is encoded by the first encoder 1690 and then provided to the first decoder 1693. In this case, the enhancement layer transform encoder 1657 provides the motion vector generated in the auxiliary information generation process to the distributed video encoding apparatus 1600.

As illustrated in FIG. 16, the transform encoding apparatus 1650 according to another embodiment of the present invention encodes a key frame reconstructed by the base layer transform encoder 1655 and then transmits the encoded key frame to the enhancement layer transform encoder 1657. And the WZ frame reconstructed by the enhancement layer transform encoder 1657 and then provided to the first decoding device 1693 to output only the result decoded by the second decoding device 1696 in the case of the base layer. In the case of the enhancement layer, the resultant images decoded by the second decoding device 1696 and the first decoding device 1693 may be rearranged and output.

In FIG. 17, the horizontal axis represents a complexity ratio of the distributed video encoding apparatus, and indicates the complexity of motion prediction. In addition, the vertical axis represents PSNR (Peak Signal to Noise Ratio) and indicates the quality of side information (SI).

Referring to FIG. 17, it can be seen that as the complexity of the distributed video encoding apparatus increases, the quality of the supplementary information is improved. When the quality of the auxiliary information is improved, the difference between the auxiliary information and the original image to be encoded is reduced, thereby reducing the parity bit required for decoding. Therefore, as the complexity of the distributed video encoding apparatus increases, the coding efficiency increases.

FIG. 18 is a graph illustrating a change in performance according to a motion prediction ratio of a distributed video encoding apparatus. The ratio of motion prediction is adjusted using TSS (Three Step Search), which is one of motion prediction methods, and the corresponding PSNR is measured. Shown.

In FIG. 18, N denotes the total number of TSS steps, and K denotes the number of TSS steps performed by the encoding apparatus.

Referring to FIG. 18, it can be seen that coding efficiency improves as the size of K increases. For example, when PSNR is 30dB, K is 2, and the coding efficiency is about 20% higher than when K is 0. When K is 6, the coding efficiency is increased by about 35% compared to when K is 0. It was. Here, when K is 2, if FEC data is additionally transmitted by the amount of bits saved compared to the case where K is 0, the packet loss rate (PLR: Packet Loss) is maintained while the total amount of bits transmitted is the same as when K is 0. Up to 20% rate can be used to transmit coded data without quality degradation.

In the distributed video encoding / decoding method and the distributed video encoding / decoding apparatus according to the embodiment of the present invention, when the channel loss occurs using the above-described characteristics, the encoding efficiency is increased by increasing the complexity of the encoding apparatus, thereby increasing the coding efficiency. The loss robustness is increased by transmitting additional FEC data by the reduced bit amount according to the efficiency.

For example, if R _a is defined as an available bit rate, r _FEC is an FEC rate, and a PLR is a packet loss rate, if _a channel loss occurs when the encoding device transmits data to the decoding device at an available bit rate Ra _a , the decoding device may Only (1-PLR) R _a is received normally. The encoder determines the FEC rate (r _FEC) using the predicted value of the available bit rate (R _a) and the packet loss rate (PLR) that indicates the channel conditions. When the FEC rate r _FEC is determined, the data transmitted by the encoding apparatus becomes (1-r _FEC × PLR) × R _a , and the FEC data becomes r _FEC × PLR × R _a . Here, the available bit rate (R _a) and the packet loss rate (PLR) is also changed according to the channel conditions due to change in accordance with the channel status r _FEC value. Thus, if the available bit-rate (R _a) is reduced thereby reducing the size of the transmittable data FEC.

In the embodiment of the present invention, in order to minimize the increase in the use of network resources due to the addition of the FEC data, the encoding apparatus is increased to increase the coding efficiency, thereby reducing the size of data to be transmitted and reducing the FEC data by the amount of the reduced data. Adding increases loss robustness without additional network resources.

19 is a flowchart illustrating a distributed video encoding method according to an embodiment of the present invention, and illustrates a process of increasing loss robustness in consideration of both available bandwidth and loss according to channel conditions.

By 19, first, the encoder obtains the available bit rate (R _a) and the packet loss rate (PLR) confirms the channel condition (Step S1910).

Thereafter, the encoding apparatus determines the FEC rate r _FEC based on the obtained packet loss rate (step S1920). For example, the FEC rate may be determined to be twice the obtained packet loss rate (PLR), but is not limited thereto and may be adjusted to a different value according to channel conditions.

Thereafter, the encoding apparatus determines the size of the video data to be transmitted and the size of the FEC data based on the packet loss rate (PLR) and the FEC rate (step S1930). Here, the size of video data to be transmitted may be determined as (1-r _FEC × PLR) × R _a , and the size of FEC data may be determined as r _FEC × PLR × R _a .

Thereafter, the encoding apparatus determines an encoding complexity for encoding according to the determined size of the video data (step S1940). The determination of the coding complexity may be performed by determining the number of execution steps of a three step search (TSS), which is one of motion prediction methods, and the determination of the number of execution steps of the TSS may be performed based on a packet loss rate in units of a predetermined image frame. It may be determined in consideration of the number of TSS execution steps compared to PLR). In addition, the number of TSS execution steps relative to the packet loss rate (PLR) may be stored in advance in the form of a lookup table, or may be obtained in real time by an arithmetic calculation.

Thereafter, the encoding apparatus performs encoding by performing motion prediction according to the encoding complexity determined in step S1940 (step S1950). Here, the encoding apparatus may provide the decoding apparatus with information such as a motion vector, a total number of TSS steps, and a number of TSS execution steps used in motion prediction.

Thereafter, the encoding apparatus performs channel encoding for compensating for channel loss, and then provides encoded data to the decoding apparatus (step S1960). Here, the encoding apparatus performs channel encoding according to the FEC rate determined in step S1920.

As shown in FIG. 19, in the distributed video encoding method according to an embodiment of the present invention, n video packets transmitted by the conventional encoding method are increased by increasing the complexity of the encoding apparatus according to channel conditions to improve encoding efficiency. By reducing the number of packets and assigning the reduced k packets to additional FEC packets, the robustness against loss can be improved without using additional network resources, thereby ensuring the quality of service.

Since steps S2010 to S2030 shown in FIG. 20 perform the same functions as those of steps S1910 to S1930 shown in FIG. 19, detailed descriptions thereof will be omitted to avoid duplication.

Referring to FIG. 20, when the size of video data to be transmitted and the size of FEC data are determined according to the channel situation through the operations of steps S2010 to S2030, the encoding apparatus adjusts the size of the video data to correspond to the determined video data size. A method for determining is determined (step S2040).

In a distributed video encoding method according to another embodiment of the present invention, a method for adjusting video size may include a method of adjusting a quantization interval, a method of increasing the complexity of an encoding apparatus, and a method of using the above two methods in combination. have.

The method of controlling the quantization interval has the advantage that there is no increase in complexity and the width of the adjustable data size is wide. However, as the size of the quantization interval increases, the quality of the video may be degraded. In addition, since the method of adjusting the complexity of the encoding apparatus performs motion prediction, the quality of the video can be kept the same, while the loss is large because the increase of the complexity limits the control of the video data size. There is a disadvantage that it is difficult to apply.

In the distributed video encoding method according to another embodiment of the present invention, a method for adjusting the quantization interval in consideration of a preset priority (for example, channel condition, performance of the encoding device, and / or user preference), encoding device The size of the video data can be adjusted by any one of a method of increasing the complexity of the method and a method of using the above two methods in combination.

For example, in the case of adjusting the size of video data using a combination of the above two methods based on the user's preference, the user can define the lowest quality reference value allowed by the user, and adjust the quality of the video by adjusting the quantization interval according to the channel situation. If the value falls below the defined reference value, the quantization interval adjustment can be stopped and the video data size can be adjusted through the complexity adjustment.

Alternatively, when high priority is given to video quality, complexity control may be applied first, and when high priority is given to usage time, that is, power consumption, quantization interval adjustment may be applied to adjust the size of video data.

As described above, when the method for adjusting the size of the video data is determined in step S2040, the encoding apparatus performs encoding based on the determined method (step S2050), and performs encoding after performing channel encoding for compensating for channel loss. The data is provided to the decoding apparatus (step S2060). Here, the encoding apparatus performs channel encoding according to the determined FEC rate.

FIG. 21 is a flowchart illustrating a distributed video encoding method according to another embodiment, and illustrates a decoding method when a distributed video encoding apparatus performs encoded motion to provide encoded data.

Referring to FIG. 21, the decoding apparatus performs channel decoding on the encoded data transmitted from the encoding apparatus to restore an erasure according to channel loss (step S2110). Here, the erasure means data lost through channel loss.

The decoding apparatus then decodes the encoded key frame provided from the encoding apparatus (step S2120). The decoding apparatus can decode the key frame through various known decoding techniques such as H.264 / AVC, MPEGx, etc., but the decoding apparatus decodes the key frame using the same standard as that used for encoding the key frame. It is preferable.

Thereafter, the decoding apparatus performs motion prediction using information such as a motion vector, a total number of TSS steps, and a number of TSS execution steps provided from the encoding device, to generate auxiliary information (SI) (step S2130).

The decoding apparatus extracts the parity bits by decoding the data from which the channel loss is restored, corrects the generated auxiliary information using the extracted parity bits (step S2140), and inversely quantizes the corrected auxiliary information to restore the WZ frame. (Step S2150).

Referring to FIG. 22, the distributed video encoding apparatus 2200 according to an embodiment of the present invention may include a keyframe encoder 2205, a channel monitor 2210, a motion predictor 2215, a transform and quantizer 2220. ), And may include a first channel encoder 2225 and a second channel encoder 2230, and encodes an input key frame to provide to a distributed video decoding apparatus, and moves a WZ frame according to a channel condition. After determining the number of prediction execution steps, motion prediction is performed according to the determined number of steps to generate parity bits, and the generated parity bits, motion vectors, and motion prediction performance information are provided to the distributed video decoding apparatus.

The keyframe encoder 2205 encodes the input keyframe and then provides the encoded keyframe to the motion predictor 2215 and the keyframe decoder 2270 of the distributed video decoding apparatus 2250. Here, the key frame encoder 2205 may encode the key frame through various known encoding techniques such as H.264 / AVC and MPEGx.

Channel monitor section 2210 is provided in the available bit rate (R _a) and / or packet loss rate channel state information after confirming the channel status by obtaining the (PLR) (packet loss rate (PLR)) a motion predictor (2215). In addition, the channel monitor 2210 determines the FEC rate based on the packet loss rate (PLR), and then provides the determined FEC rate to the second channel encoder 2230.

The motion predictor 2215 determines the number of motion prediction execution steps (that is, the number of execution steps K of the TSS) based on the channel condition information (packet loss rate PLR) provided from the channel monitor unit 2210, and determines the determined motion prediction. Based on the number of steps performed, motion prediction is performed on the encoded key frame to generate auxiliary information, and the generated auxiliary information is provided to the transform and quantization unit 2220. In addition, the motion prediction unit 2215 may use the auxiliary information generation unit of the distributed video decoding apparatus 2270 to transmit information such as a motion vector (MV) used for motion prediction and the total number of TSS steps and the number of TSS steps that are motion prediction information. 2260).

The transform and quantizer 2220 transforms and quantizes auxiliary information provided from the motion predictor 2215 and provides the transformed quantizer 2220 to the first channel encoder 2225.

The first channel encoder 2225 generates a parity bit for the quantized data using the channel code and provides the parity bit to the second channel encoder 2230.

The second channel encoder 2230 performs channel encoding to compensate for erasure due to channel loss in accordance with the FEC rate provided from the channel monitor unit 2210, and then distributes the data on which channel encoding is performed. The decoding apparatus 2270 is provided. The second channel encoder 2265 may perform channel encoding using, for example, a Reed-Solomon code.

If the PLR provided by the channel monitor unit 2210 is included in the allowable range in the distributed video encoding apparatus 2200, motion prediction may not be performed. In such a case, the transformed and quantized unit may be used to convert the input WZ frame. 2220.

In addition, the distributed decoding apparatus 2250 according to an embodiment of the present invention may include a keyframe decoder 2255, an auxiliary information generator 2260, a second channel decoder 2265, and a first channel decoder 2270. And an image reconstructor 2275, which decodes an encoded keyframe provided from the distributed video decoding apparatus 2200, and performs motion prediction on the decoded keyframe using the provided motion vector and motion prediction information. After generating the auxiliary information, the WZ frame is recovered by removing noise included in the generated auxiliary information using the channel-decoded parity bit.

In detail, the keyframe decoder 2255 decodes an encoded keyframe provided from the keyframe encoder 2205 of the distributed video encoding apparatus 2200.

The auxiliary information generator 2260 generates the auxiliary information by performing motion prediction on the decoded key frame using information such as the motion vector, the TSS total number of steps, and the number of TSS execution steps provided from the distributed video encoding apparatus 2225. do. In this case, the auxiliary information generated by the auxiliary information generator 2260 may be the same as the auxiliary information generated by the motion predictor 2215 since the auxiliary information is generated using the motion prediction information provided from the distributed video encoding apparatus 2200. have.

The second channel decoder 2265 channel-decodes the data provided from the distributed video encoding apparatus 2200 to reconstruct the eraser according to the channel loss.

The first channel decoder 2270 extracts a parity bit by performing channel decoding on the reconstructed data, and then estimates the quantized data by correcting the noise included in the auxiliary information using the extracted parity bit.

The image reconstructor 2255 reconstructs the WZ frame by inversely quantizing the data quantized estimated by the first channel decoder 2270 and performing inverse transformation. In this case, ambiguity generated during inverse quantization may be solved by referring to auxiliary information provided from the auxiliary information generator 2260.

Referring to FIG. 23, a distributed video encoding apparatus 2300 according to another embodiment of the present invention may include a keyframe encoder 2305, a controller 2330, a motion predictor 2310, a transform and quantizer 2315, The first channel encoder 2320 and the second channel encoder 2325 may be included.

The keyframe encoder 2305 encodes the input keyframe and then provides the encoded keyframe to the motion predictor 2310 and the keyframe decoder 2355 of the distributed video decoding apparatus 2350. Here, the key frame encoder 2355 may encode the key frame through various well-known encoding techniques such as H.264 / AVC and MPEGx.

The controller 2330 may include a channel monitor 2333 and a resource manager 2336. Channel monitor section 2333 is an available bit rate (R _a) and the packet loss rate after confirming the channel status by obtaining the (PLR), obtained the available bit rate (R _a) and the packet loss rate (PLR) the resource management unit (2336) to provide.

A resource management unit (2336) is FEC rate (r _FEC) after setting the FEC rate (r _FEC), is set on the basis of the available bit rate (R _a) and the packet loss rate (PLR) as provided by the channel monitor section 2333 to the second The channel encoder 2325 is provided. Here, the resource manager 2336 may set the FEC rate r _FEC to twice the packet loss rate PLR.

In addition, the resource management unit 2336 calculates the size of the video data ((1-r _FEC × PLR) × R _a ) and the size of the FEC data (r _FEC × PLR × R _a ), and calculates the calculated data size and a preset value. The video data scaling method is determined based on the priority.

The resource manager 2336 may adjust the number of motion prediction performance steps (ie, the number of performance steps K of the TSS) or the quantization interval to correspond to the size of the calculated video data, and determine to perform the motion prediction and the quantization interval control in combination It may be. In this case, the resource manager 2336 adjusts the quantization interval in consideration of the channel condition, the performance of the encoding apparatus, and / or the user's preference, the motion prediction means step, and any one of a combination of the above two methods. After determining the method as a method of adjusting the size of the video data, a control signal corresponding to the determined may be provided.

The motion prediction unit 2310 performs motion prediction on the encoded key frame based on the motion prediction execution step number (that is, the execution step number K of the TSS), which is a control signal provided from the resource management unit 2336, and receives auxiliary information. The generated auxiliary information is provided to the transform and quantization unit 2315. In addition, the motion predictor 2310 may provide information such as a motion vector (MV) used for motion prediction and information such as the total number of TSS steps, the number of TSS steps, and the number of TSS execution steps, which are motion prediction information, to the auxiliary information generator of the distributed video decoding apparatus 2350 ( 2360).

The transformer and quantizer 2315 transforms and quantizes auxiliary information provided from the motion predictor 2310 and provides the transformed quantizer 2315 to the first channel encoder 2320. When the method for adjusting the size of video data is determined by adjusting the quantization interval, the transform and quantization unit 2315 may perform quantization corresponding to the quantization interval, which is a control signal provided from the resource manager 2336.

The first channel encoder 2320 generates a parity bit for the quantized data using the channel code and provides the parity bit to the second channel encoder 2325.

The second channel encoder 2325 performs channel encoding to compensate for erasure due to channel loss, and then provides the distributed video decoding apparatus 2350 with the data subjected to channel encoding. Here, the second channel encoder 2370 may perform channel encoding according to the FEC rate information provided from the resource manager 2336.

In addition, the distributed video decoding apparatus 2350 according to another embodiment of the present invention may include a keyframe decoder 2355, an auxiliary information generator 2360, a second channel decoder 2370, and a first channel decoder 2380. And an image restoration unit 2390.

The keyframe decoder 2355 decodes the encoded keyframe provided from the keyframe encoder 2305 of the distributed video encoding apparatus 2300.

The auxiliary information generator 2360 generates auxiliary information by performing motion prediction on the decoded key frame using information such as a motion vector, a TSS total number of steps, and a number of TSS execution steps provided from the distributed video encoding apparatus 2300. do. The auxiliary information generated by the auxiliary information generator 2360 may be the same as the auxiliary information generated by the motion predictor 2310 since the auxiliary information generated by the auxiliary video encoding apparatus 2300 is generated using the motion prediction information provided from the distributed video encoding apparatus 2300. have.

The second channel decoder 2370 reconstructs the erasure according to the channel loss by channel decoding the data provided from the distributed video encoding apparatus 2300.

The first channel decoder 2380 extracts a parity bit by performing channel decoding on the reconstructed data of the eraser and then estimates the quantized data by correcting the noise included in the auxiliary information using the extracted parity bit.

The image reconstructor 2390 reconstructs the WZ frame by performing inverse quantization and inverse quantization of the quantized data estimated from the first channel decoder 2380. Here, the ambiguity generated during inverse quantization may be solved by referring to the auxiliary information provided from the auxiliary information generator 2360.

According to the embodiments of the present invention, the above-described distributed video encoding method may be used in a plurality of image capturing apparatuses (distributed video encoding apparatuses), and data provided by the plurality of distributed video encoding apparatuses is decoded by the plurality of distributed video decoding apparatuses. Can be.

That is, a plurality of distributed video encoding apparatuses and distributed video decoding apparatuses according to an embodiment of the present invention may be provided.

Hereinafter, various embodiments in which the distributed video encoding apparatus and the distributed video decoding apparatus using the distributed video encoding and decoding method are utilized will be described.

In the following embodiments of the present invention, the distributed video encoding apparatus may be used in the same meaning as an image photographing apparatus and a camera.

For convenience of description, a detailed configuration of an image capturing apparatus that is a distributed video encoding apparatus and a distributed video decoding apparatus that decodes data provided by the image capturing apparatus will not be described. However, an image capturing apparatus or a camera according to an embodiment of the present invention will be described below. In the embodiment of the present invention, the image capturing apparatus, the camera, and the distributed video encoding apparatus are used in the same sense.) In the decoding apparatus for decoding the data transmitted from the image capturing apparatus or the camera apparatus, the above-described distributed video encoding apparatus and distributed video are described. The configuration of the decoding device can be used.

For example, a component for capturing an image, such as an image capturing apparatus or a camera, according to an embodiment of the present invention may include a distributed video encoding apparatus and an encoding method using distributed video coding according to an embodiment of the present invention. To encode an image. For example, the image capturing apparatus may include a WZ frame encoder and a key frame encoder to encode image information.

In addition, as described above, the image capturing apparatus may further include a buffer or a motion compensation unit in addition to the WZ frame encoder and the keyframe encoder to perform motion prediction using information provided from a distributed video decoder or a decoder.

In addition, as described above, the image capturing apparatus or the camera checks the state of the channel capable of transmitting data from the image capturing apparatus to the distributed video decoding apparatus, and determines the channel coding rate and the size of the video data to be transmitted based on the identified channel condition. The channel monitor may be further included.

The distributed video decoding apparatus decodes a plurality of image data provided from an image capturing apparatus or a camera, and includes a keyframe decoder, a channel code decoder, an auxiliary information generator, and an image reconstruction unit, which are components included in the aforementioned distributed video decoding apparatus. It may include.

In addition, the distributed video decoding apparatus further includes an additional encoder to encode distributed video coded images in a general video decoding apparatus by encoding data decoded by the distributed video decoding apparatus using a general encoding method using a general encoding method rather than a distributed video encoding method. Enable decryption.

In the following embodiments of the present invention, an apparatus for performing distributed video encoding will be described with an image capturing apparatus and a camera as an example. However, a configuration in which the apparatus for performing distributed video encoding is provided outside the image capturing apparatus and the camera is also provided. It is included in the scope of right.

In the following embodiment of the present invention, in the case of an embodiment using a plurality of image capturing apparatus, it is assumed for convenience of description that all of the plurality of image capturing apparatuses perform distributed video encoding, but only some of the plurality of image capturing apparatuses are used. A method of performing distributed video coding and using general video coding other than distributed video coding may be used.

Referring to FIG. 24, in order to perform distributed video coding using the plurality of image capturing apparatuses 2400, a plurality of image capturing apparatuses 2400 may perform distributed video coding and images provided from a plurality of image capturing apparatuses 2400. A metadata tagging unit 2410 for assigning attribute information to the distributed video decoding apparatus 2420 and a distributed video decoding apparatus 2420 for decoding information provided from the plurality of image capturing apparatuses 2400 may be included.

According to an embodiment of the present invention, only a part of the plurality of image capturing apparatuses 2400 may perform distributed video coding, and other methods may use general video coding instead of distributed video coding. It is assumed that both the imaging apparatus 2400 perform distributed video coding on the images provided through the imaging units of the plurality of imaging apparatuses 2400 using distributed video coding.

The image capturing apparatus 2400 may include a distributed video encoding apparatus and encode an image using an encoding method using distributed video coding according to an embodiment of the present invention. For example, the image capturing apparatus 2400 may include a WZ frame encoder and a key frame encoder to encode image information.

As described above, the image capturing apparatus 2400 may further include a buffer or a motion compensator in addition to the WZ frame encoder and the key frame encoder to perform motion prediction using information provided from the distributed video decoder. In addition, as described above, the image capturing apparatus 2400 checks the state of a channel capable of transmitting data from the image capturing apparatus to the distributed video decoding apparatus 2420, and thus the channel encoding rate and the video to be transmitted are determined based on the identified channel condition. The channel monitor unit for determining the data size may be further included.

The metadata tagging unit 2410 is a part capable of assigning attribute information related to data to a plurality of image data photographed by the image capturing apparatus 2400, and provides location information of the image capturing apparatus 2400, information related to a photographing target, Various attribute information such as photographing object movement information and color characteristics of an image may be provided to the plurality of image data photographed by the image capturing apparatus 2400. The metadata is data provided to a plurality of video data contents, and may selectively decode a plurality of videos generated based on metadata provided to the plurality of video data.

According to the exemplary embodiment of the present invention, the metadata tagging unit 2410 is represented as an independent device for convenience of description, but when it is not included in the image capturing apparatus or it is not necessary to give attribute information to the image data, the metadata tagging unit 2410 ) May not be used.

The distributed video decoding apparatus 2420 is a portion that decodes a plurality of image data provided from the image capturing apparatus 2400, and includes a keyframe decoder, a channel code decoder, and an auxiliary component included in the distributed video decoding apparatus 2420. It may include an information generating unit and an image restoring unit.

In addition, the distributed video decoding apparatus 2420 includes an additional encoder to decode an image in a general video decoding apparatus by encoding data decoded by the distributed video decoding apparatus using a general encoding method using a general encoding method instead of a distributed video encoding method. To be able.

Referring to FIG. 24, it is shown that the plurality of image capturing apparatus 2400 photographs a playground in which a soccer game occurs. Some of the plurality of image capturing apparatuses 2400 photograph individual players, some of the entire stadium, and some of the plurality of image capturing videos of the goal situation from behind the goal post, each camera captures a video of various views according to the role. The metadata tagging unit 2410 tags metadata to multiple video streams input from the plurality of image capturing apparatuses 2400.

According to an embodiment of the present invention, the viewer may further include a component for selecting a view captured by a specific image capturing apparatus or a component for providing an image for selectively viewing a specific event based on tagged metadata. May selectively provide image data corresponding to a desired view to the user, or additionally provide a component that selectively provides only a specific event desired by the user.

Referring to FIG. 25, the media sharing system includes a media generating apparatus including a plurality of image capturing apparatuses 2500 and a media generating server 2510, and one or more clients 2520 connected to the media generating apparatus through a communication network.

Each of the image capturing apparatuses 2500 is provided with different unique identifiers (GUIDs), and photographs images of different viewpoints and provides them to the media generation server 2510. The image capturing apparatus 2500 may be a camera for capturing an image.

At least one of the image capturing apparatuses 2500 may include a distributed video encoder to encode an image captured by the image capturer using distributed video encoding. The image capturing apparatus 2500 may be connected to the media generation server 2510 by wire or wirelessly. The image capturing apparatus 2500 and the media generation server 2510 may be connected through an IEEE 1394 network or a wireless network.

The media generation server 2510 may include a distributed video decoding apparatus and may decode image data transmitted from the image capturing apparatus 2500. In some cases, the distributed video decoding apparatus may be provided at the client 2520 side.

The media generation server 2510 allocates the same multicast address and different port numbers to each image photographed by the plurality of photographing apparatuses 2500, and includes a session description including an assigned multicast address and a port number. Protocol) message is generated and transmitted to the plurality of clients 2520. At this time, the media generation server 2510 receives images from the plurality of photographing apparatuses by executing one VP producer, and assigns the same multicast address and different port numbers to the plurality of received images.

In addition, the media generation server 2510 receives an SDP message and receives a multicast address different from the multicast address allocated through the SAP (Service Access Point) packet to the client 2520 through a communication network. Here, the SDP message includes service attribute information including at least one of an assigned multicast address and a plurality of port numbers, a producer's name, an e-mail address, a video codec, a video quality, and a service type.

The client 2520 analyzes the SDP message from the media generating device 2510 to extract service attribute information, and the image by the photographing apparatus 2500 selected by the user or the multi-view image by the plurality of photographing apparatuses 2500. Outputs

In addition, the client 2520 may display the multi-view image by the plurality of photographing apparatuses 2500 as a wide image using the network tiled display device.

26 to 27 illustrate a configuration of an apparatus for photographing a stereoscopic image using distributed video encoding and decoding according to an embodiment of the present invention, which illustrates a specific embodiment of a system for photographing a stereoscopic image. The stereoscopic image photographing system using the distributed video encoder or the distributed video decoder according to the above-described embodiment of the present invention may also be included in the scope of the present invention.

As shown in Fig. 26, the three-dimensional object tracking device according to the present invention is a frame of an image acquired at the same time from at least two (preferably three or more) video cameras (multi-view camera) 2600 At least one depth map acquired at the same time from a multi-view video storage unit 2610 storing data and at least two (preferably three or more) depth cameras 2605. depth map storage unit 2620 for storing map data, camera information including a camera focal length of each viewpoint, a camera correction unit 2630 for obtaining a base matrix of positions and directions between viewpoints, and camera information An object region extracting unit 2640 for extracting a moving object region from the background in the image using a matrix matrix, a multiview image of the previous frame and the current frame, and a depth map data; The object motion estimation unit 2650 for detecting a motion object of the body region and obtaining three-dimensional motion vectors of the object, and detecting a shielding region in the object region, and then performing the backward motion estimation of the previous frame and the current frame. And a shielding area processor 2660 for estimating the position of the obscured object. The apparatus may further include a multiview image corrector 2635 for correcting the epipolar line of the image acquired by the multiview camera 2600 to coincide using the base matrix. The apparatus further includes an object motion display unit 2670 for displaying the extracted object motion on the screen. The apparatus may further include a multiview camera control and driver 2680 for adjusting the position of the multiview camera 2600 such that the object is located at the center region in the image in case the object deviates from the image.

The multiview camera 2600 may encode the image photographed through the image pickup unit through distributed video encoding using the distributed video encoder included in the multiview camera 2600. Distributed video encoding performed by the multiview camera 2600 may use the same method as the above-described method.

The multi-view video storage unit 2610 and the depth map storage unit 2620 store video images obtained from several video cameras and one depth camera in a data storage device such as a video tape or a frame capture device. The stored image information is used as an input by the camera corrector 2630, the multiview image corrector 2635, the object region extractor 2640, and the object motion estimator 2650.

The multi-view video storage unit 2610 and the depth map storage unit 2620 may be connected to a plurality of video cameras and one depth camera by a wired or wireless network.

The distributed video decoder may be provided at a portion of the multiview video storage unit 2610 to process the image, and may decode the distributed video encoded image generated by the multiview camera 2600.

The camera corrector 2630 extracts feature points between various images that know the distance from the camera 2600, and then obtains a rotation matrix and a movement matrix for mutual conversion using the correspondence between the feature points and the actual distance information. . In addition, camera information such as focal length is obtained through a camera correction technique. This information is used to continue object tracking by using a camera at another point in time when the object motion estimation unit 2650 does not generate the shielding area when the shielding area is generated.

The multi-view image corrector 2635 uses the camera transformation matrix (base matrix) obtained by the camera corrector 2630 to refer to the epipolar line of the image acquired by the remaining cameras with respect to the reference camera. Make it parallel to the rows of the image. This allows you to reduce the search area and time for object tracking and stereo matching.

The object region extractor 2640 extracts the pixels of the object region based on a criterion of the object and the background pixel using the change in brightness and depth of the sequentially input current image and the previously input image, and includes all of the pixels. A bounding box is placed on the current or next frame, and the coordinates of the start and end points are stored in the buffer. This is used to find the motion vector in the frame following the object motion.

The object motion estimator 2650 compares the object region of the current frame extracted by the object region extractor 2640 with the object region of the next frame, and then compares the object region of the next frame with the smaller row and column size in the next frame. By selecting and moving the points in the detected block area toward the center of the block, the position of the block having the highest similarity is detected. At this time, the motion vector between the center points of each block is detected as a motion vector in the x and y directions. At this time, when the similarity is smaller than the predetermined threshold or when the matching is performed backward, the center of the center point is regarded as that there is no shielding area in the block only when the position difference from the original block center is within a certain number of pixels. The motion vector is obtained, and if not, the corresponding area is detected by the shielding area processor 2660 as the shielding area.

Referring to FIG. 27, a multiview image processing apparatus includes a multiview camera 2700, a depth camera 2710, a three-dimensional warping unit 2720, a trimap generator 2730, an alphamat generator 2740, A foreground extractor 2750 is included.

The multi-view camera 2700 is composed of a plurality of cameras (first to n-th cameras, 2700-1, 2700-2, and 2700-n) as shown. The viewpoints of the plurality of cameras are different from each other according to the position of the camera, and thus a plurality of images having different viewpoints are bundled together to be called multiview images. The multi-view image obtained from the multi-view camera 2700 includes color information for each pixel of the 2D image forming each image, but does not include depth information of the 3D image.

The depth camera 2710 acquires a depth map having depth information in three dimensions. The depth camera 2710 is a device capable of illuminating a laser or an infrared ray to an object or a target area, obtaining a return beam, and obtaining depth information in real time. The depth camera 2710 includes a depth sensor that senses depth information through a laser or infrared ray.

At least one of the multiview camera 2700 and the depth camera 2710 may include a distributed video encoding apparatus. Images captured by the image capture unit using the distributed video encoding apparatus included in the multiview camera 2700 and the depth camera 2710 may be encoded, and the encoded video data may be a trimap generator 2730 or a 3D warping unit 2720. ) And may be decoded by a distributed video decoding apparatus that may be included in the ramimap generator 2730 or the 3D warping unit 2720 or converted into another image format.

Image data provided by the multiview camera 2700 and the depth camera 2710 may be provided to the trimap generator 2730 or the 3D warping unit 2720 using wired or wireless.

Referring back to FIG. 27, the multiview image obtained from the multiview camera 2700 and the multiview depth map from the 3D warping unit 2720 are input to the trimap generator 2730, and the trimap generator 2730 is provided. ) Generates a multiview trimap consisting of trimaps corresponding to each of the images forming the multiview image using the multiview depth map.

The multiview trimap from the trimap generator 2730 is input to the alphamat generator 2740, and the alphamat generator 2740 uses the input multiview trimap to provide a multiview alpha corresponding to the multiview image. You can create a matte.

The multiview alphamat generated from the alphamat generator 2740 is input to the foreground extractor 2750, and the foreground extractor 2750 applies a multiview alphamat to the multiview image from the multiview camera 2700. Multiview foreground can be extracted.

The trimap generator 2730 uses a target image and a three-dimensional warped depth map to generate a trimap corresponding to each image forming a multiview image.

The trimap generator 2730 may include a segment part, a foreground area determiner, a hole remover, and an erosion / expansion calculator.

The segment unit generates segmented images by performing color segmentation on each image forming a multiview image. Color segmentation refers to a process of dividing an image into regions (segments) of similar colors.

The segment unit sets segments between pixels whose luminance or chrominance difference between adjacent pixels is less than or equal to a threshold in the target image to which color segmentation is to be performed.

In the depth map obtained by the depth camera 2710, the depth value does not exist at all or is very sparse in an area far from the depth camera 2710 according to the setting of the depth camera 2710, and the depth camera 2710 In the area nearer to), all depth values exist or are very dense. The same is true of the three-dimensional warped depth map, except that a hole, which is a pixel whose depth value is not defined, is generated due to the three-dimensional warping. As described above, since the discontinuity point of the depth value may occur at the boundary of the segment, it may be determined that the region having a large depth value among the segments is the foreground region, and the other region is the background region.

Therefore, the foreground area determiner segments each of the three-dimensional warped depth maps corresponding to the segmented image by the segment unit, and sets the foreground area of the set of segments having a depth value greater than or equal to a certain ratio in the segment. Decide on The ratio may be arbitrarily determined appropriately, for example, to determine the foreground area as a set of segments having a depth value of 70% or more.

The erosion / expansion calculator generates an immersion trimap by applying erosion and expansion operations to the input multiview depth map. The trimap is the image before obtaining the alpha matte, which is essential for matting, divided into the foreground and the background, and the unknown region whether it is the foreground or the background. The erosion / expansion calculating unit generates each image of the depth map constituting the input multiview depth map as a binary image according to a predefined threshold value, and then erodes the edge of the binary image inward and extends it out to generate an unknown region. This creates a multiview trimap.

There are a plurality of holes whose depth values are not defined in the three-dimensional warped depth map, and the hole removing unit fills the holes with appropriate depth values in the foreground area of the three-dimensional warped depth map using surrounding pixel values. Parts outside the foreground area are no longer of interest, so you can only fill the hall with the foreground area.

Referring to FIG. 28, when the monitored person appears in the monitoring area of the camera 2800, the motion is detected by the motion sensor mounted on the low power wireless camera and the camera is immediately operated. In this case, the information captured by the camera is compressed to be transmitted in the wireless sensor network, and the compressed data is transmitted to the gateway 2820 through the wireless sensor network. The image data transmitted to the gateway 2820 is transmitted to the server 2830 through an Ethernet network, and the image data transmitted to the server 2830 is viewed through the monitoring system 2840.

Since the low power wireless camera 2800 operates in the existing sensor network, the low power wireless camera 2800 must be operated by a battery in the same manner as the operation of the sensor nodes in the existing sensor network system. Therefore, it is based on low power operation due to the limitation of power supply capacity. However, since the video is basically based on continuous shooting, the motion monitoring sensor is mounted on the camera 2800 to minimize the power consumption by operating the camera 2800 only when the motion of the monitoring area is detected.

The camera 2800 includes a distributed video encoding apparatus capable of performing distributed video encoding, and the distributed video encoding apparatus may encode an image captured by the camera through distributed video encoding.

The low power wireless camera 2800 may include a battery, a micro controller unit (MCU), a motion detection sensor, a distributed video encoding apparatus, and a wireless communication module.

The motion detection sensor may be installed at the front of the wireless camera 2800 to detect a motion of a person in an area to be monitored. At this time, the detected signal is notified to the MCU. The camera 2800 functions to capture an image of the surveillance area. In response to a signal detected by the motion detection sensor and transmitted to the MCU, the MCU applies power to the camera 2800 to start recording. If no movement is detected for a certain period of time, shooting ends. In the distributed video encoding apparatus, an image captured by the wireless camera 2800 may be encoded through distributed video encoding.

Wireless signals are compressed to less than or equal to 1M bandwidth. The MCU controls the battery and the wireless communication module, receives the motion detection from the motion detection sensor, and determines the operation of each part and applies power. The wireless communication module wirelessly transmits distributed video encoded images in a distributed video encoding apparatus and is in charge of communication with a wireless sensor network. The battery supplies power to each part under the control of the MCU.

The same scheme as described above can also be used in, for example, a baby monitoring system for monitoring a home using a wireless camera that encodes an image captured using distributed video coding indoors.

Referring to FIG. 29, a surveillance system using a plurality of cameras 2900 may form three-dimensional data using a plurality of cameras 2900.

As illustrated in FIG. 29, a surveillance system using a plurality of cameras 2900 according to the present invention is connected to a plurality of cameras 2900 and a plurality of cameras 2900, which are installed in a plurality of monitored areas, from a camera 2900. A central monitoring unit 2910 is configured to receive captured image data and to control an operation of each camera 2900.

The plurality of cameras 2900 may encode an image photographed through the imaging unit through distributed video encoding using a distributed video encoder included in the camera 2900. Distributed video encoding performed by the camera 2900 may use the same method as described above.

The central monitoring apparatus 2910 of the present invention includes a GIS database 2920 for displaying geographic information of a region to be monitored as 3D image data, and an actual image screen captured by the camera 2900 and a GIS database 2920. Using the 3D image data stored in) to generate a three-dimensional image that is synchronized with the camera screen to describe the camera screen in the form of three-dimensional graphics, calculates the position of a specific point or the monitored object on the camera screen in three-dimensional coordinates On the basis of the calculated 3D coordinate values, a plurality of cameras may be linked to photograph a specific point or a monitoring object.

The method of acquiring 3D coordinates of a camera image using 3D spatial data constructs 3D spatial data for providing a 3D image of an area to be monitored. Calculate the change of angle of view according to the zoom change of the camera, synchronize the screen of the camera with the screen of the 3D image, calculate the 3D coordinates for a specific point of the actual camera screen, and then use the 3D coordinates of the camera image using the 3D spatial data. The 3D coordinates of the image may be obtained by using a method of obtaining.

According to one embodiment of the present invention, the forest fire detection can be used as an example using a wireless image pickup device including a sensor other than the sensor for detecting the forest fire.

The plurality of wireless image capturing apparatuses 3000 are installed in place of loading in a wide area such as a mountain or a national park for detecting a fire and tracking a fire. Since the wireless image capturing apparatus 3000 according to the present invention is a wireless device, it is easier to install in a mountainous region than a general image camera or CCTV which should be connected by wire.

The wireless image capturing apparatus 3000 may include a distributed video encoding apparatus capable of performing distributed video encoding, and may encode an image captured by using distributed video encoding according to an embodiment of the present invention. Image data encoded through distributed video encoding may be provided to a distributed video decoder to be decoded.

The image data generated by the wireless image capturing apparatus 3000 is collected by the gateway 3010 and passed through a router 3020 to a surveillance or control center in charge of forest fire detection and ignition tracking via a communication modem 3030 through a communication network. Can be sent.

Since the data of the pre-installed points are secured in each of the wireless video image pickup apparatuses 3000, the control center can accurately know the point of imaging. Therefore, using the unique number already assigned to each wireless image capturing apparatus 3000 and the installed point matching table, it is possible to clearly grasp the state of the point reflected by the wireless image capturing apparatus 3000. When appearing in the imaging device 3000 it is also possible to accurately determine the location of the missing person. The control server (not shown) of the control center may determine the occurrence of fire through the change of the image while periodically receiving the image generated by the wireless image pickup device 3000.

The fire detector may detect the fire from the CCTV and the thermal imaging camera, but may determine the fire from the detection signal of the fire detection sensor 3040. Alternatively, the fire detector may be located at a control center to determine the fire.

A plurality of fire detection sensors 3040 may be employed to facilitate the occurrence of fire. The fire detection sensor 3040 may use a sensor that detects temperature, smoke, or flame, but is not necessarily limited thereto. Any sensor capable of detecting a fire may be used. The sensor may also use a fire detection sensor that senses one or more combinations of temperature, smoke, or flame.

The fire sensor 3040 may be installed separately from the wireless image capturing apparatus 3000 or may be installed in combination with the wireless image capturing apparatus 3000. Fire detection sensing signal can be transmitted to the control server of the control center, and in addition to the location of the fire detection sensor 3040 will be easier to determine in which position the fire occurred.

When the fire is detected through the fire detection sensor 3040, the control server of the control center can determine the location of the fire, and if the location is detected, the position of the surrounding wireless video imaging device 3000 to capture the location of the fire. Can be fixed selectively. By doing so, it is possible to secure more imaging data of the fire occurrence point.

FIG. 31 is a block diagram illustrating a fake face detection apparatus using an infrared image according to an exemplary embodiment of the present invention. The image acquirer 3110, the face region extractor 3120, the fake face detector 3130, and the face recognizer are shown. 3140.

The image acquisition unit 3110 includes an infrared image acquisition unit 3113 and a general image acquisition unit 3116. The infrared image acquisition unit 3113 acquires an infrared image captured by the infrared camera 3103 and provides the infrared image to the forged face detection unit 3130. The general image obtaining unit 3116 obtains a general image captured by the general camera 3106 and provides the same to the face region extracting unit 3120.

The infrared camera 3103 and the general camera 3106 may be provided with a distributed video encoder, and distributed video encoding may be performed using the distributed video encoder. The distributed video encoded data by the infrared camera 3103 and the general camera 3106 may be wirelessly transmitted to a specific server, and the server may determine whether the user is based on the transmitted data.

The face region extractor 3120 is a component-based technique using an image processing technique with respect to a general image input from the general image acquirer 3116, or a pattern recognition technique of Adaboost or SVM (Support Vector Machine, SVM). Extracts a face region and provides it to the forged face detector 3130.

The fake face detector 3130 includes a face image evaluator, a face / no face discrimination unit, and an eyeball characteristic analyzer for evaluating the image quality characteristics of the infrared image. The face image evaluator evaluates the image quality of the infrared image extracted by the face region extractor 3120 and coincides with the input face region by using the technique of Sharpness, and as a result of the evaluation, the sharpness of the infrared image is lower than that of the face region. It detects with a fake face and provides a detection result of the fake face to the face recognition unit 3140. On the other hand, if the face region and the infrared image coincide with each other, the fake face is detected as not a fake face. The detection result is provided to the face recognition unit 3140.

The face presence / non-determination unit is based on the characteristics of the infrared image having the characteristic that the shape of the face is not visualized in the case of the infrared image of the photograph or the fake face, and the face region extracted by the face region extraction unit 3120 is input. Using the technology of Adaboost or SVM to discriminate the presence or absence of a face, if the face is absent, it is detected as a fake face, and the detection result of the fake face is provided to the face recognition unit 3140, If the face is unique, it is detected as not a fake face, and a detection result of not being a fake face is provided to the face recognizing unit 3140.

The eyeball characterization unit detects and presets an eyeball area in the actual infrared image of the user in advance, and then analyzes between the predetermined eyeball areas and the eyeball area of the infrared image input from the infrared image acquisition unit 3113 and analyzes the eyeball area. As a result, when the predetermined eye areas and the eye area of the infrared image do not have the same thing, the detected face is detected as a fake face, and the detection result of the fake face is provided to the face recognition unit 3140. If any of the eyeballs and the eyeballs of the infrared image are the same, it is detected as not a fake face, and the detection result of not detecting the fake face is provided to the face recognizing unit 3140.

The face recognition unit 3140 performs face recognition on the face region input from the face region extraction unit 3120 according to the evaluation result, the determination result, and the analysis result of not being a fake face input from the fake face detection unit 3130. On the other hand, the face recognition unit 3140 does not perform face recognition according to the evaluation result and the determination result and the analysis result of the forgery face input from the fake face detection unit 3130, or the infrared camera 3103 and the general camera 3106. Control to retry shooting.

Accordingly, the present invention captures a face image by using an infrared / visible light camera at the same time, and analyzes the characteristics of the dual input infrared image to detect whether or not a fake face, and in the case of non-falsified face, the image captured by the visible light By performing user authentication using images, it is difficult to distinguish real face information from fake face information by using a common face recognition method as in the past, the limitations of using expensive equipment, Many problems that make you feel uncomfortable can be solved.

24 to 31 illustrate some embodiments in which distributed video encoding according to an embodiment of the present invention is used, and when distributed video encoding according to the above-described embodiment of the present invention can be used. However, the present invention is not limited to the above-described embodiment and may be utilized in various forms.

Referring to FIG. 32, the distributed video encoding apparatus 3200 may include a plurality of distributed video encoders 3200-1, 3200-2, and 3200-n, and the distributed video decoding apparatus may include a plurality of distributed video decoders ( 3210).

For convenience of description, each of the distributed video encoders 3200-1, 3200-2, and 3200-n and the distributed video decoder 3210 are shown as separate components, but a plurality of distributed video encoders 3200 and a plurality of distributed units are shown. The video decoder 3210 may have an integrated configuration that shares some configurations.

In addition, for convenience of description, each encoder is represented by an encoder that performs distributed video encoding. However, the encoder may have a plurality of distributed video encoders and may use a general encoder that does not perform distributed video encoding.

Each distributed video encoder 3200 and distributed video decoder 3210 may have a configuration of the distributed video encoder 3200 described above.

That is, the distributed video encoder 3200 may include a keyframe encoder, a quantizer, a block unitizer, and a channel code encoder having the above-described configuration. In addition, the distributed video encoder 3200 may include a motion compensator, a first converter, and a quantization controller. , A second transform unit, a motion predictor, a channel monitor, and the like.

Similarly, the distributed video decoder 3210 may include a keyframe decoder, an auxiliary information generator, a channel code decoder, and an image reconstructor, and may further include a picture buffer, a motion vector generator, a first encoder, It may include a second encoder.

In the above, the present invention has been described in detail with reference to preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications and changes by those skilled in the art within the spirit and scope of the present invention. This is possible.

Claims

A distributed video decoding apparatus using variable block motion prediction,

A key frame decoder for reconstructing a key frame transmitted from the key frame encoding apparatus;

A motion vector generator configured to determine a size of a block for performing motion prediction using the reconstructed keyframe, generate a motion vector, and output the generated motion vector;

An auxiliary information generator configured to generate auxiliary information using the reconstructed key frame and the generated motion vector;

A channel code decoder for estimating a quantized value using parity bits transmitted from a distributed video encoding apparatus and the auxiliary information; and

And an image restoring unit for restoring a current frame to be distributed video decoding based on the quantized value and the auxiliary information estimated by the channel decoding unit.
The method of claim 1, wherein the motion vector generator,

A plurality of blocks having different sizes of blocks are determined for motion estimation of the auxiliary information, motion vectors between the reconstructed key frames are obtained based on the determined blocks, and motion among the obtained motion vectors is determined based on a predetermined criterion. Distributed video decoding apparatus for selecting a vector.
The apparatus of claim 2, wherein the motion vector generator

A block size determiner which determines a plurality of block sizes for motion prediction using the reconstructed key frame;

A second block that most closely matches the first block by searching an area of another key frame of the restored key frames to find an area that matches the first block of one of the restored key frames A motion prediction execution unit for finding a motion vector and generating a motion vector for each of the plurality of block sizes, the motion vector being a difference value between the first block and the second block;

A motion vector storage unit for storing motion vectors generated by the motion prediction execution unit; And

A motion vector selector configured to measure a similarity between the first block of the one key picture and the second block of the other key picture among the motion vectors, and select a motion vector having a block size corresponding to the highest similarity; Distributed video decoding device.
The method of claim 3, wherein the motion vector selector,

Sum of Absolute Difference (SAD), Mean Absolute Difference (MAD), and Sum of Square Difference (SSD) between the first block and the second block Distributed video decoding apparatus for measuring the similarity using any one of (square sum of the difference of the blocks).
The size of the block for performing the motion prediction,

And at least one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 block sizes.
The method of claim 5, wherein the size of the block for performing the motion prediction,

Distributed video decoding apparatus characterized in that determined by determining the change in motion between the reconstructed key frames.
The method of claim 6, wherein the block size determiner,

And determining a block size for performing the motion prediction to at least one of block sizes of 16x16, 16x8, and 8x16 when the motion change between the reconstructed key frames is small.
The method of claim 6, wherein the block size determiner,

And determining a block size for performing the motion prediction to at least one of block sizes of 8x8, 8x4, 4x8, and 4x4 when the motion change between the reconstructed key frames is large.
A distributed video decoding method using variable block motion prediction,

Restoring a key frame transmitted from the key frame encoding apparatus;

Generating a motion vector by determining a block size for performing motion prediction using the reconstructed key frame;

Generating auxiliary information using the reconstructed key frame and the generated motion vector;

Estimating a quantized value using parity bits transmitted from a distributed video encoding apparatus and the auxiliary information; And

And reconstructing a current frame to be distributed video decoding by using the estimated quantized value and the auxiliary information.
The method of claim 9, wherein the determining of the block size for performing motion prediction using the reconstructed key frame to generate a motion vector comprises:

Determining a plurality of blocks having different sizes for motion estimation of the auxiliary information;

Obtaining motion vectors between the reconstructed keyframes based on the determined plurality of blocks; And

And selecting a motion vector among the obtained motion vectors based on a predetermined criterion.
The method of claim 10, wherein the obtaining of the motion vectors between the reconstructed keyframes based on the determined plurality of blocks comprises:

A second block that most closely matches the first block by searching an area of another key frame of the restored key frames to find an area that matches the first block of one of the restored key frames Finding out; And

Generating a motion vector for each of the plurality of block sizes, the motion vector being a difference between a first block and a second block.
The method of claim 10, wherein the selecting of the motion vector among the obtained motion vectors based on the predetermined criterion includes:

Measuring similarity between the first block of one of the reconstructed key frames and the second block of the other one of the reconstructed key frames with respect to the respective motion vectors; And

And selecting a motion vector between the first block and the second block having the highest similarity among the similarities of the first block and the second block measured for each motion vector.
The method of claim 12, wherein the similarity is,

Sum of Absolute Difference (SAD) between the first block and the second block, Mean Absolute Difference (MAD), and Sum of Square Difference (SSD) of each block Distributed video decoding method using any one of the following methods.
The method of claim 9, wherein the size of the block for performing the motion prediction,

And at least one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 block sizes.
The method of claim 14, wherein the size of the block for performing the motion prediction,

Distributed video decoding method, characterized in that determined by determining the change in motion between the reconstructed key frames.
The method of claim 15, wherein the block size determiner,

And when the motion change between the reconstructed key frames is small, determining a size of a block for performing the motion prediction to at least one of block sizes of 16x16, 16x8, and 8x16.
The method of claim 15, wherein the block size determiner,

And determining a block size for performing the motion prediction to at least one of 8x8, 8x4, 4x8, and 4xx block sizes when the motion change between the reconstructed key frames is large.
A distributed video encoding method for encoding an input image divided into a first frame and a second frame,

Encoding the first frame and transmitting an encoded first frame;

Receiving a motion vector from a distributed video decoding apparatus;

Generating auxiliary information corresponding to the second frame based on the provided motion vector;

Obtaining a prediction error of the auxiliary information based on the generated auxiliary information and the second frame; And

And quantizing the second frame based on the obtained prediction error.
The method of claim 18, wherein the distributed video encoding method,

Transforming the input second frame;

After the auxiliary information is generated, converting the generated auxiliary information; And

And after the second frame is quantized, generating parity bits by performing channel encoding on the quantized second frame.
The method of claim 19, wherein the obtaining of the prediction error of the auxiliary information based on the generated auxiliary information and the second frame comprises:

And obtaining the prediction error based on the transformed auxiliary information and the transformed second frame.
The method of claim 19, wherein the obtaining of the prediction error of the auxiliary information based on the generated auxiliary information and the second frame comprises:

Comparing the converted auxiliary information with the converted second frame to determine a position of a block in which an error occurs;

Determining an amount of error in the converted auxiliary information; And

And obtaining a crossover probability based on the determined amount of error and a length of an encoding code.
The method of claim 21, wherein quantizing the second frame based on the obtained prediction error comprises:

Determining the number of quantization bits based on the obtained prediction error; And

And determining whether to perform quantization of each block constituting the second frame based on the obtained prediction error.
The method of claim 22, wherein determining whether to perform quantization of each block constituting the second frame based on the obtained prediction error comprises:

An error does not occur for every block constituting the second frame, and a block whose cross probability is less than a preset threshold is omitted from quantization, and a block in which an error occurs or the cross probability is greater than or equal to the threshold is quantized. Distributed video encoding method, characterized in that for executing.
Decoding the encoded first frame;

Generating a motion vector based on the at least one decoded first frame;

Transmitting the generated motion vector to an encoding device;

Generating auxiliary information corresponding to a second frame by using the generated motion vector;

Correcting the auxiliary information using the parity bit and the auxiliary information provided from the encoding apparatus; And

Inversely quantizing the corrected auxiliary information to reconstruct the second frame.
A distributed video encoding apparatus for encoding an input image divided into a first frame and a second frame,

An encoder which encodes the first frame;

A motion compensator configured to generate auxiliary information corresponding to the second frame based on the motion vector provided from the decoding apparatus and the encoded first frame;

A quantization controller configured to obtain a prediction error of the auxiliary information based on the generated auxiliary information and the input second frame, and to quantize the second frame based on the obtained prediction error; And

And a channel encoder configured to channel-encode the quantized second frame to generate parity bits.
The apparatus of claim 25, wherein the distributed video encoding apparatus is

A first transforming unit transforming the generated auxiliary information;

And a second converter configured to transform the input second frame.
The method of claim 26, wherein the quantization control unit

And the prediction error is obtained based on the transformed auxiliary information and the transformed second frame.
The method of claim 26, wherein the quantization control unit

Comparing the transformed auxiliary information and the transformed second frame to determine the position of the block in which an error occurs, determine the amount of error in the transformed auxiliary information, the amount of the error and the encoding code And obtaining the prediction error by obtaining a crossover probability based on a length.
The method of claim 28, wherein the quantization control unit

And determining the number of quantization bits based on the obtained prediction error, and determining whether to perform quantization of each block constituting the second frame.
The method of claim 29, wherein the quantization control unit

For every block constituting the second frame, an error does not occur and a block whose cross probability is smaller than a preset threshold is omitted. A block in which an error occurs or the cross probability is greater than or equal to the threshold is quantized. Distributed video encoding device, characterized in that the execution.
A decoder which decodes the encoded first frame;

An auxiliary information generator configured to generate a motion vector based on the at least one decoded first frame, provide the generated motion vector to an encoding apparatus, and generate auxiliary information corresponding to a second frame using the motion vector;

A conversion unit for converting the generated auxiliary information;

A channel decoder to correct the auxiliary information based on the converted auxiliary information and the parity bit provided from the encoding apparatus; And

And a dequantizer for reconstructing the second frame by dequantizing the corrected auxiliary information.
A decoder which decodes the encoded first frame;

An auxiliary information generator configured to generate a motion vector based on the at least one decoded first frame, provide the generated motion vector to an encoding apparatus, and generate auxiliary information corresponding to a second frame using the motion vector;

A conversion unit for converting the generated auxiliary information;

A channel decoder to correct the auxiliary information based on the converted auxiliary information and the parity bit provided from the encoding apparatus;

An inverse quantizer for restoring the second frame by inversely quantizing corrected auxiliary information; And

And a coding unit for encoding the reconstructed second frame.
33. The apparatus of claim 32, wherein the encoder

And encoding the reconstructed second frame and the first frame decoded by the decoder.
33. The apparatus of claim 32, wherein the encoder

A first encoder to encode the reconstructed second frame; And

And a second encoder which encodes the first frame decoded by the decoder.
In the encoding method of the distributed video encoding apparatus,

Checking channel status;

Determining a channel coding rate and a size of video data to be transmitted based on the identified channel condition;

Determining a number of steps of performing motion prediction based on the determined size of the video data to be transmitted;

Encoding the video data to be transmitted by performing motion prediction according to the determined number of motion prediction steps; And

And channel coding the encoded video data according to the determined channel coding rate.
36. The method of claim 35, wherein identifying the channel condition comprises:

Distributed video encoding method characterized by obtaining an available bit rate and a packet loss rate (PLR).
37. The method of claim 36, wherein determining the channel coding rate and the size of video data to transmit based on the identified channel conditions,

And a size of the video data to be transmitted and a size of the channel coded data based on the available bit rate, the packet loss rate, and the channel code rate.
The method of claim 35, wherein the determining of the number of motion prediction performing steps based on the determined size of the video data to be transmitted comprises:

And determining the number of motion prediction performing steps among the total steps of a three step search (TSS) based on the determined size of video data to be transmitted.
36. The method of claim 35, wherein encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps comprises:

And providing the distributed video decoding apparatus with the motion vector and the number of motion prediction performance steps used for the motion prediction.
In the encoding method of the distributed video encoding apparatus,

Checking channel status;

Determining a channel coding rate and a size of video data to be transmitted based on the identified channel condition;

Determining at least one method of adjusting the number of motion prediction steps and adjusting the quantization interval based on the determined size of the video data to be transmitted as a method for adjusting the size of video data;

Performing encoding of video data to be transmitted according to the determined video data size adjusting method; And

And channel coding the encoded video data according to the determined channel coding rate.
41. The method of claim 40, wherein the determining of at least one method of adjusting the number of motion prediction steps and adjusting the quantization interval based on the determined size of the video data to be transmitted as the method for adjusting the video data size comprises:

Determining at least one of the motion prediction performance step adjustment and the quantization interval adjustment as a method for adjusting the video data size based on at least one prioritized priority among channel conditions, user preferences, and performance of the encoding apparatus. Distributed video encoding method, characterized in that.
In the distributed video decoding method,

Recovering channel loss by channel decoding the provided encoded second data;

Restoring a first frame by decoding the provided encoded first data;

Generating auxiliary information by performing motion prediction on the reconstructed first frame based on the provided motion vector and the number of motion prediction performing step information;

Correcting the assistance information generated using parity bits; And

Dequantizing the corrected auxiliary information to reconstruct the second frame.
A first frame encoder which encodes the input first frame;

A channel monitor unit for obtaining a packet loss rate (PLR) and determining a channel coding rate based on the obtained packet loss rate;

A motion prediction unit for determining the number of motion prediction execution steps based on the provided packet loss rate, and generating auxiliary information by performing motion prediction according to the determined number of motion prediction execution steps; And

And a second frame encoder configured to generate the parity bits by encoding the generated auxiliary information and perform channel encoding according to the determined channel coding rate.
44. The apparatus of claim 43, wherein the channel monitor unit

And determining twice the packet loss rate as a forward error correction (FEC) rate.
The method of claim 43, wherein the second frame encoder

A transform and quantizer for converting and quantizing the generated auxiliary information;

A first channel encoder configured to channel-code the transformed and quantized data to generate parity bits; And

And a second channel encoder for performing channel encoding on the data provided from the first channel encoder according to the channel encoding rate.
44. The apparatus of claim 43, wherein the motion predictor

A distributed video encoding apparatus, comprising: providing a distributed video decoding apparatus with a motion vector and motion prediction performing step number information used for motion prediction.
A first frame encoder which encodes the input first frame;

A controller for determining a channel coding rate and a size of video data to be transmitted based on a packet loss rate and an available bit rate, and providing a control signal for adjusting the size of video data based on the determined size of the video data to be transmitted;

A motion predictor configured to generate auxiliary information by performing motion prediction when a control signal is provided from the controller; And

And a second frame encoder configured to generate the parity bits by encoding the generated auxiliary information and perform channel encoding according to the determined channel coding rate.
48. The method of claim 47, wherein the control unit

Based on the determined size of the video data to be transmitted, at least one of the number of motion prediction execution steps and the quantization interval adjustment is determined as a method for adjusting the video data size, and a control signal corresponding to the method for adjusting the determined video data size is determined. Distributed video encoding device, characterized in that provided.
49. The apparatus of claim 48, wherein the motion predictor

Distributed video encoding, wherein the motion prediction is performed according to the number of motion prediction execution steps, which are control signals provided from the control unit, and the motion vector and the motion prediction execution step information used for the motion prediction are provided to the distributed video decoding apparatus. Device.
The method of claim 48, wherein the second frame encoder

A conversion and quantization unit of the generated auxiliary information;

A first channel encoder configured to channel-code the transformed and quantized data to generate parity bits; And

And a second channel encoder for performing channel encoding on the data provided from the first channel encoder according to the channel encoding rate.
51. The method of claim 50, wherein the transform and quantization unit

And if quantization interval information is provided from the controller, quantization is performed according to the provided quantization interval information.
In the encoding method of a plurality of distributed video encoding apparatuses,

Performing distributed video encoding on the captured image using the plurality of distributed video encoding apparatuses; And

Transmitting the video data on which the distributed video encoding has been performed to a distributed video decoder through a predetermined communication channel,

Distributed video encoding performed on at least one of the plurality of distributed video encoding apparatuses may include:

A distributed video encoding method for encoding an input image divided into a first frame and a second frame,

Encoding the first frame and transmitting an encoded first frame;

Receiving a motion vector from a distributed video decoding apparatus;

Generating auxiliary information corresponding to the second frame based on the provided motion vector;

Obtaining a prediction error of the auxiliary information based on the generated auxiliary information and the second frame; And

And quantizing the second frame based on the obtained prediction error.
The method of claim 52, wherein the distributed video encoding method is

Transforming the input second frame;

After the auxiliary information is generated, converting the generated auxiliary information; And

And after the second frame is quantized, generating parity bits by performing channel encoding on the quantized second frame.
The method of claim 53, wherein the obtaining of the prediction error of the auxiliary information based on the generated auxiliary information and the second frame comprises:

And obtaining the prediction error based on the transformed auxiliary information and the transformed second frame.
The apparatus of claim 52, wherein the plurality of distributed video encoding apparatuses are

At least one of the plurality of distributed video encoding apparatuses may be used as a depth image capturing apparatus for obtaining depth information of a subject, and the plurality of distributed video encoding apparatuses except the depth image capturing apparatus may photograph the subject at a plurality of viewpoints, And obtaining a 3D image of the subject based on the depth information provided from the depth image photographing apparatus and the image information associated with the subject photographed at the plurality of viewpoints.
The apparatus of claim 52, wherein the plurality of distributed video encoding apparatuses are

At least one of the plurality of distributed video encoding apparatus comprises a predetermined sensor for detecting whether there is a fire, and transmits the image data photographed by the plurality of distributed video encoding apparatus and the fire determination data provided by the sensor using a wireless network Distributed video encoding method.
The apparatus of claim 52, wherein the plurality of distributed video encoding apparatuses are

The individual images captured by capturing individual images according to a plurality of photographing viewpoints are decoded by a distributed video decoding method to be provided as a multi-view screen having a plurality of screens, or only a part of the photographed images are provided by a specific control signal. Distributed video encoding method, characterized in that.
The apparatus of claim 52, wherein the plurality of distributed video encoding apparatuses are

When a monitored person appears in a surveillance area, motion is detected by motion sensors mounted on the plurality of distributed video encoding apparatuses, and the distributed video encoding apparatus starts to operate, and distributed video encoding is decoded to the plurality of distributed video encoding apparatuses. The compressed data is compressed to be transmitted in a wireless sensor network, and the compressed data is transmitted and received to a router through a wireless sensor network through a wireless communication module included in the plurality of distributed video encoding apparatuses, and then transmitted and received from the router to a gateway. Distributed video encoding apparatus characterized in that the image data transmitted to the gateway is transmitted and received to the server via the Ethernet network.
The apparatus of claim 52, wherein the plurality of distributed video encoding apparatuses are

Obtain an infrared image of a predetermined subject using at least one of the plurality of distributed video encoding apparatuses as an infrared image photographing apparatus, and compare the image quality sharpness of the image captured with at least one of the plurality of distributed video encoding apparatuses with the infrared image. And detecting whether the object is counterfeit.
In the video encoding method using a plurality of video recording apparatus,

Performing distributed video encoding on an image photographed using a distributed video encoder included in the plurality of image capturing apparatuses; And

Transmitting the video data on which the distributed video encoding has been performed to a distributed video decoder through a predetermined communication channel,

Distributed video encoding performed on at least one of the plurality of distributed video encoding apparatuses may include:

Checking channel status;

Determining a channel coding rate and a size of video data to be transmitted based on the identified channel condition;

Determining at least one method of adjusting the number of motion prediction steps and adjusting the quantization interval based on the determined size of the video data to be transmitted as a method for adjusting the size of video data;

Performing encoding of video data to be transmitted according to the determined video data size adjusting method; And

And channel coding the encoded video data according to the determined channel coding rate.
61. The method of claim 60, wherein identifying the channel condition comprises:

Distributed video encoding method characterized by obtaining an available bit rate and a packet loss rate (PLR).
61. The method of claim 60, wherein the determining of the number of motion prediction performing steps based on the determined size of the video data to be transmitted comprises:

And determining the number of motion prediction performing steps among the total steps of a three step search (TSS) based on the determined size of video data to be transmitted.
61. The method of claim 60, wherein encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps comprises:

And providing the distributed video decoding apparatus with the motion vector and the number of motion prediction performance steps used for the motion prediction.
61. The apparatus of claim 60, wherein the plurality of distributed video encoding apparatuses are

At least one of the plurality of distributed video encoding apparatuses may be used as a depth image capturing apparatus for obtaining depth information of a subject, and the plurality of distributed video encoding apparatuses except the depth image capturing apparatus may photograph the subject at a plurality of viewpoints, And obtaining a 3D image of the subject based on the depth information provided from the depth image photographing apparatus and the image information associated with the subject photographed at the plurality of viewpoints.
61. The apparatus of claim 60, wherein the plurality of distributed video encoding apparatuses are

At least one of the plurality of distributed video encoding apparatus comprises a predetermined sensor for detecting whether there is a fire, and transmits the image data photographed by the plurality of distributed video encoding apparatus and the fire determination data provided by the sensor using a wireless network Distributed video encoding method.
61. The apparatus of claim 60, wherein the plurality of distributed video encoding apparatuses are

The individual images captured by capturing individual images according to a plurality of photographing viewpoints are decoded by a distributed video decoding method to be provided as a multi-view screen having a plurality of screens, or only a part of the photographed images are provided by a specific control signal. Distributed video encoding method, characterized in that.
61. The apparatus of claim 60, wherein the plurality of distributed video encoding apparatuses are

When a monitored person appears in a surveillance area, motion is detected by motion sensors mounted on the plurality of distributed video encoding apparatuses, and the distributed video encoding apparatus starts to operate, and distributed video encoding is decoded to the plurality of distributed video encoding apparatuses. The compressed data is compressed to be transmitted in a wireless sensor network, and the compressed data is transmitted and received to a router through a wireless sensor network through a wireless communication module included in the plurality of distributed video encoding apparatuses, and then transmitted and received from the router to a gateway. Distributed video encoding apparatus characterized in that the image data transmitted to the gateway is transmitted and received to the server via the Ethernet network.
61. The apparatus of claim 60, wherein the plurality of distributed video encoding apparatuses are

Obtain an infrared image of a predetermined subject using at least one of the plurality of distributed video encoding apparatuses as an infrared image photographing apparatus, and compare the image quality sharpness of the image captured with at least one of the plurality of distributed video encoding apparatuses with the infrared image. And detecting whether the object is counterfeit.