WO2011084037A2 - Procédé et appareil de codage/décodage vidéo distribué et appareil de transcodage - Google Patents

Procédé et appareil de codage/décodage vidéo distribué et appareil de transcodage Download PDF

Info

Publication number
WO2011084037A2
WO2011084037A2 PCT/KR2011/000194 KR2011000194W WO2011084037A2 WO 2011084037 A2 WO2011084037 A2 WO 2011084037A2 KR 2011000194 W KR2011000194 W KR 2011000194W WO 2011084037 A2 WO2011084037 A2 WO 2011084037A2
Authority
WO
WIPO (PCT)
Prior art keywords
distributed video
frame
auxiliary information
motion
channel
Prior art date
Application number
PCT/KR2011/000194
Other languages
English (en)
Korean (ko)
Other versions
WO2011084037A3 (fr
Inventor
전병우
김도형
서덕영
김철근
심동규
민경연
박시내
Original Assignee
성균관대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020100002080A external-priority patent/KR101074919B1/ko
Priority claimed from KR1020100012743A external-priority patent/KR101059318B1/ko
Priority claimed from KR1020100026598A external-priority patent/KR101103974B1/ko
Application filed by 성균관대학교산학협력단 filed Critical 성균관대학교산학협력단
Priority to US13/521,552 priority Critical patent/US9210436B2/en
Publication of WO2011084037A2 publication Critical patent/WO2011084037A2/fr
Publication of WO2011084037A3 publication Critical patent/WO2011084037A3/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/395—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability involving distributed video coding [DVC], e.g. Wyner-Ziv video coding or Slepian-Wolf video coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115—Selection of the code volume for a coding unit prior to coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/89—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder

Definitions

  • the present invention relates to image encoding and decoding, and more particularly, to a distributed video encoding / decoding method, a distributed video encoding / decoding apparatus, and a transform encoding apparatus that can be applied to a distributed video coding technique.
  • MPEG, H.26x, and other compression standards are widely used as efficient compression technologies for video players, customized video information services (VOD), video telephony, digital multimedia broadcasting (DMB), and video transmission in wireless mobile environments.
  • the compression standards have a large gain in coding efficiency by eliminating temporal redundancy.
  • As a representative method for reducing the temporal redundancy there are motion prediction and compensation techniques.
  • the motion prediction and compensation technique requires a relatively large amount of computation in the video encoder, power consumption increases. Therefore, in a limited resource environment such as a sensor network, in order to reduce the power of the encoder, reducing the complexity of the encoder has emerged as an important technical problem.
  • DVC Distributed video coding
  • the Slepian-Wolf theory mathematically proves that even if the correlated sources are encoded independently, the decoding gains can be equally obtained by performing the prediction encoding on each source together.
  • the Wyner-Ziv theory extends the Slepian-Wolf theory, which is a lossless compression, to lossy compression.
  • Wyner-Ziv coding technology is a representative method of distributed video coding technology, and generates side information of a current frame by using similarity between neighboring frames reconstructed by a decoding apparatus, and a difference between the generated auxiliary information and the current frame.
  • the distributed video encoding technique reduces the complexity of the encoder by allowing the decoder to perform motion prediction, which takes up the largest amount of computation in the encoder.
  • the encoder encodes video frames independently of each other, Since the video frames are not scanned to detect similarity, the amount of computation of the encoder can be reduced.
  • one of the most important parts in terms of coding efficiency is the prediction accuracy of the auxiliary information corresponding to the WZ frame in the decoder and the error probability of channel estimation in parity channel decoding. to be.
  • the information used to generate parity bits through channel encoding in the distributed video encoder is an original WZ frame
  • the information used when performing channel decoding using parity bits transmitted from the distributed video decoder is predicted WZ. Since the information is auxiliary information corresponding to the frame, the higher the similarity between the original WZ frame and the auxiliary information, the more the amount of parity bits used for channel decoding can be reduced, thereby reducing the amount of computation required.
  • FIG. 1 is a diagram illustrating a configuration of an encoder 110 and a decoder 130 corresponding to the conventional Wyner-Ziv coding technique.
  • the encoder 110 classifies pictures of source video content into two types. One is a frame to be encoded by using a distributed video coding method (hereinafter referred to as a 'WZ frame'), and the other is a picture to be encoded by a conventional coding method (hereinafter referred to as a 'key frame') rather than a distributed video coding method. )to be.
  • a 'WZ frame' a distributed video coding method
  • a 'key frame' a conventional coding method
  • the keyframes are encoded by the intraframe encoding method of, for example, H.264 / AVC in the keyframe encoder 114 and transmitted to the decoder 130.
  • the keyframe decoder 133 of the decoder 130 corresponding to the encoder 110 according to the conventional Wyner-Ziv coding technique reconstructs the transmitted keyframes.
  • the auxiliary information generating unit 134 generates side information corresponding to the WZ frame by using the key frame restored by the key frame decoding unit 133 and converts the auxiliary information to the channel code decoding unit 131.
  • the auxiliary information generator 134 assumes linear motion between key frames located before and after the current WZ frame, and generates side information corresponding to the WZ frame to be restored using interpolation.
  • the interpolation method may be used instead of the interpolation method, but since the noise in the auxiliary information generated by the interpolation method is smaller than the noise in the auxiliary information generated by the interpolation method, the interpolation method is used in most cases.
  • the quantization unit 112 of the encoder 110 performs quantization on the WZ frame and outputs the quantization value of the WZ frame to the block unitization unit 112.
  • the block unit unit 111 classifies the quantized value of the input WZ frame into predetermined coding units.
  • the channel code encoder 113 generates a parity bit for each coding unit by using the channel code.
  • the generated parity bits are temporarily stored in a parity buffer (not shown), and are sequentially transmitted when the decoder 130 requests parity through a feedback channel.
  • the channel code decoder 131 of FIG. 1 receives the parity transmitted from the encoder 110 and estimates the quantized value.
  • the image restorer 132 of FIG. 1 receives the quantized value estimated by the channel code decoder 131 and inversely quantizes the quantized value to restore the WZ frame.
  • the decoder 131 requests the encoder 110 to transmit more parity bits through a feedback channel. This process is repeated until decryption succeeds.
  • the distributed video encoding method using the feedback channel has the advantage of accurately performing channel modeling due to the above-described structure, but has a disadvantage in that the quality of service decreases due to a large transmission delay according to the feedback. .
  • a distributed video coding method that does not use a feedback channel (eg, PRISM: Power-efficient, Robust high-compression, Syndrome-based Multimedia Coding) performs channel modeling through a classifier of an encoding device and thereby uses a bit rate. Control is performed and success of decryption is checked through a cyclic redundancy check (CRC).
  • PRISM Power-efficient, Robust high-compression, Syndrome-based Multimedia Coding
  • the accuracy of the channel modeling is improved, but the complexity of the coding apparatus increases, thereby reducing the advantages of the distributed video coding method. If the motion compensation is not performed, the accuracy of channel modeling decreases, thereby reducing the compression efficiency.
  • the channel encoding method is a method of additionally transmitting a Forward Error Correction (FEC) packet when a channel loss occurs
  • the retransmission method is a method of retransmitting a packet having a loss.
  • FEC Forward Error Correction
  • both the method using channel coding and the method using retransmission have to transmit a larger amount of data than the original data to be transmitted, which leads to an increase in the use of network resources, thereby limiting its use when network resources are limited.
  • the distributed video encoding method based on the Wyner-Ziv theory is a technique of reconstructing a current WZ frame by correcting noise added to auxiliary information generated by a decoder using parity. Therefore, the less noise added to the generated auxiliary information, the smaller the amount of parity required. Therefore, it is important to generate auxiliary information well without noise in order to have good performance in terms of rate distortion.
  • the conventional auxiliary information generating method estimates a motion vector using a fixed size block (eg, 8x8) existing in the reconstructed key picture and reconstructs the motion vector of the auxiliary information to be restored in consideration of the distance between frames. Obtained from motion vectors between keyframes.
  • the decoding unit in the key frame indicated by the motion vector of the auxiliary information thus obtained is generated as auxiliary information.
  • FIG. 2 illustrates a conventional generation of auxiliary information using an interpolation method.
  • FIG. 3 illustrates a conventional generation of auxiliary information using extrapolation.
  • the motion vector of the auxiliary information to be generated is obtained from the motion vector using a fixed size block existing in the reconstructed keyframe, the motion is complicated between frames or the motion does not linearly change or an object And when the background suddenly disappears or appears, incorrect auxiliary information can be generated.
  • the size of the fixed block used for motion prediction between key frames increases, the accuracy of the obtained motion vector is similar to the motion of an actual image in a simple region such as a background, but not in a part where a complex object exists.
  • 8x8 is generally used for the block size used for motion prediction for generating auxiliary information.
  • a first object of the present invention is to provide a distributed video decoding apparatus using variable block motion prediction.
  • a second object of the present invention is to provide a distributed video decoding method using variable block motion prediction.
  • a fifth object of the present invention is to provide a transform encoder that encodes an image encoded by the distributed video encoding method so that it can be decoded by a general decoder.
  • a sixth object of the present invention is to provide a distributed video encoding / decoding method capable of improving loss robustness and quality of service.
  • a seventh object of the present invention is to provide a distributed video encoding / decoding apparatus capable of improving loss robustness and quality of service.
  • An eighth object of the present invention is to provide an image encoding method using a plurality of image photographing apparatuses.
  • a ninth object of the present invention is to provide an image encoding method using a plurality of image photographing apparatuses.
  • a distributed video decoding apparatus using variable block motion prediction for achieving the first object of the present invention includes a keyframe decoder for reconstructing a keyframe transmitted from a keyframe encoding apparatus, and the reconstructed key.
  • a motion vector generator for generating a motion vector by determining a size of a block for performing a motion prediction using a frame and outputting the generated motion vector, and supplementary information using the reconstructed key frame and the generated motion vector.
  • An auxiliary information generator to generate, a channel code decoder to estimate a quantized value using the parity bits transmitted from the distributed video encoding apparatus and the auxiliary information, and the quantized value and the auxiliary information estimated by the channel decoder Copy the current frame to be distributed video decoding based on Image restoration may include a to.
  • the motion vector generator determines a plurality of blocks having different block sizes for motion prediction of the auxiliary information, obtains motion vectors between the reconstructed key frames based on the determined blocks, and obtains the motion vectors based on a predetermined criterion.
  • a motion vector may be selected among the motion vectors.
  • the motion vector generator is a block size determiner that determines a plurality of block sizes for motion prediction using the reconstructed key frame, and finds an area that matches a first block of one key frame among the reconstructed key frames. The region of one of the reconstructed key frames is searched to find a second block that most closely matches the first block, and the motion vector is a difference value between the first block and the second block.
  • a motion prediction execution unit for generating a plurality of block sizes, a motion vector storage unit for storing the motion vectors generated by the motion prediction execution unit, and a first block of the one key picture among the motion vectors; The similarity between the second blocks of the other key picture is measured to obtain a motion vector of a block size corresponding to the highest similarity.
  • the motion vector selector includes a sum of absolute difference (SAD) between the first block and the second block, a mean absolute difference (MAD), and an average sum of SSDs.
  • the degree of similarity may be measured using any one of (of: Square Difference).
  • the block size for performing the motion prediction may be at least one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 block sizes.
  • the size of the block for performing the motion prediction may be determined by determining a change in motion between the reconstructed key frames.
  • the block size determiner may determine a block size for performing the motion prediction to at least one of the block sizes of 16x16, 16x8, and 8x16 when the change in motion between the reconstructed key frames is small.
  • the block size determiner may determine a block size for performing the motion prediction with at least one of block sizes of 8x8, 8x4, 4x8, and 4x4 when the motion change between the reconstructed key frames is large.
  • Generating a motion vector by determining a block size for performing motion prediction using the reconstructed key frame may include determining a plurality of blocks having different sizes for motion prediction of the auxiliary information, and determining the plurality of determined blocks.
  • the method may include obtaining motion vectors between the reconstructed keyframes and selecting a motion vector among the obtained motion vectors based on a predetermined criterion.
  • Obtaining motion vectors between the reconstructed keyframes based on the determined plurality of blocks comprises: selecting among the reconstructed keyframes to find a region that matches a first block of one of the reconstructed keyframes.
  • Selecting a motion vector among the obtained motion vectors according to the predetermined criterion may include a first block of one key frame of the reconstructed key frames and a second key of the other one of the reconstructed key frames. Measuring a similarity between blocks for each motion vector and selecting a motion vector between the first block and the second block having the highest similarity among the similarities of the first block and the second block measured for each motion vector; It may include the step.
  • the degree of similarity may include the sum of absolute difference (SAD) between the first block and the second block, the mean absolute difference (MAD), and the sum of square difference (SSD). The sum of squares of the differences of each block).
  • the block size for performing the motion prediction may be at least one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, and 4x4 block sizes.
  • the size of the block for performing the motion prediction may be determined by determining a change in motion between the reconstructed key frames.
  • the block size determiner may determine a size of a block for performing the motion prediction with at least one of block sizes of 16x16, 16x8, and 8x16 when the change in motion between the reconstructed key frames is small.
  • the block size determiner may determine a size of a block for performing the motion prediction with at least one of block sizes of 8x8, 8x4, 4x8, and 4x4 when a change in motion between the reconstructed key frames is large.
  • a distributed video encoding method for encoding an image divided into a first frame and a second frame according to an aspect of the present invention for achieving the third object of the present invention includes encoding and encoding the first frame. Transmitting one frame, receiving a motion vector from a distributed video decoding apparatus, generating auxiliary information corresponding to the second frame based on the provided motion vector, the generated auxiliary information and the second frame Obtaining a prediction error of the auxiliary information based on the quantization of the second frame based on the obtained prediction error.
  • the distributed video encoding method may include transforming the input second frame, generating the auxiliary information, transforming the generated auxiliary information, and after the second frame is quantized, The method may further include generating parity bits by performing channel encoding on the second frame. Obtaining the prediction error of the auxiliary information based on the generated auxiliary information and the second frame may obtain the prediction error based on the converted auxiliary information and the converted second frame. Obtaining a prediction error of the auxiliary information based on the generated auxiliary information and the second frame may be performed by comparing the converted auxiliary information and the converted second frame to determine a location of a block in which an error has occurred.
  • the method may include determining an amount of an error in the transformed auxiliary information, and obtaining a crossover probability based on the determined amount of the error and a length of an encoding code.
  • Quantizing the second frame based on the obtained prediction error includes determining a number of quantization bits based on the obtained prediction error and configuring each second frame based on the obtained prediction error. And determining whether to execute quantization of the block. Determining whether to execute quantization of each block constituting the second frame based on the obtained prediction error does not occur for each block constituting the second frame, and the cross probability is determined in advance. Blocks smaller than the set threshold may omit quantization, and blocks having an error or having a cross probability greater than or equal to the threshold may perform quantization.
  • a distributed video encoding apparatus configured to encode an input image divided into a first frame and a second frame, the encoder configured to encode the first frame;
  • a motion compensator for generating auxiliary information corresponding to the second frame based on the motion vector provided from the decoding apparatus and the encoded first frame, and the auxiliary information based on the generated auxiliary information and the input second frame.
  • the apparatus may include a quantization control unit that obtains a prediction error, quantizes the second frame based on the obtained prediction error, and a channel encoder that generates parity bits by channel coding the quantized second frame.
  • the distributed video encoding apparatus may further include a first transformer configured to transform the generated auxiliary information and a second transformer configured to transform the input second frame.
  • the quantization controller may obtain the prediction error based on the transformed auxiliary information and the transformed second frame.
  • the quantization control unit compares the transformed auxiliary information with the transformed second frame to determine a location of a block in which an error occurs, determines an amount of an error in the converted auxiliary information, and determines the amount of the error. And by obtaining a crossover probability based on a length of an encoding code, the prediction error may be obtained.
  • the quantization controller may determine the number of quantization bits based on the obtained prediction error and determine whether to perform quantization of each block constituting the second frame.
  • the quantization control unit does not generate an error for each block constituting the second frame and omits quantization for a block whose cross probability is less than a preset threshold, and an error occurs or the cross probability is greater than or equal to the threshold.
  • the block may perform quantization.
  • a distributed video decoding apparatus including: a decoder configured to decode an encoded first frame and a motion vector based on at least one decoded first frame.
  • An auxiliary information generator for providing the generated motion vector to an encoding apparatus and generating auxiliary information corresponding to a second frame using the motion vector, a transformer for converting the generated auxiliary information, and converted auxiliary information
  • a channel decoder to correct the auxiliary information based on the parity bits provided from the encoding apparatus, and an inverse quantizer to dequantize the corrected auxiliary information to restore the second frame.
  • a transform encoding apparatus including: a decoder configured to decode an encoded first frame and a motion vector based on the decoded at least one first frame.
  • An auxiliary information generator for providing the generated motion vector to an encoding apparatus and generating auxiliary information corresponding to a second frame using the motion vector, a transformer for converting the generated auxiliary information, converted auxiliary information, and A channel decoder to correct the auxiliary information based on the parity bits provided from the encoding apparatus, an inverse quantizer to dequantize the corrected auxiliary information to restore the second frame, and an encoder to encode the restored second frame can do.
  • the encoder may encode the reconstructed second frame and the first frame decoded by the decoder.
  • the encoder may include a first encoder that encodes the reconstructed second frame and a second encoder that encodes the first frame decoded by the decoder.
  • the encoding method of the distributed video encoding apparatus comprises the steps of: identifying a channel condition, and determining a channel encoding rate and a size of video data to be transmitted based on the identified channel condition. Determining, based on the determined size of the video data to be transmitted, determining the number of motion prediction steps, encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps, and determining And channel encoding the encoded video data according to a channel coding rate.
  • the checking of the channel condition may obtain an available bit rate and a packet loss rate (PLR).
  • the determining of the channel coding rate and the size of the video data to be transmitted based on the identified channel situation may include determining the size of the video data to be transmitted and the size of the channel encoding data based on the available bit rate, the packet loss rate, and the channel coding rate. Can be.
  • the determining of the number of motion prediction performing steps based on the determined size of the video data to be transmitted may determine the number of motion prediction performing steps among the total steps of TSS (Three Step Search) based on the determined size of the video data to be transmitted.
  • Encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps further includes providing the distributed video decoding apparatus with motion vector and motion prediction performance step information used for motion prediction. It may include.
  • the method may include: checking a channel condition, determining a channel coding rate and a size of video data to be transmitted based on the identified channel condition, Determining at least one method of adjusting the number of steps and adjusting the quantization interval based on the size of the video data as a method for adjusting the video data size, and encoding the video data to be transmitted according to the determined video data size adjusting method. And performing channel encoding on the encoded video data according to the determined channel coding rate.
  • Determining at least one method of adjusting the number of motion prediction steps and adjusting the quantization interval based on the determined size of the video data to be transmitted is a method for adjusting the size of video data.
  • At least one of the motion prediction performance step adjustment and the quantization interval adjustment may be determined as a method for adjusting video data size based on at least one prioritized priority among performances.
  • the method includes reconstructing channel loss by channel decoding the provided encoded second data, and reconstructing the first frame by decoding the provided encoded first data. Generating auxiliary information by performing motion prediction on the reconstructed first frame based on the provided motion vector and information on the number of steps of performing motion prediction; correcting the auxiliary information generated using parity bits; And inversely quantizing the supplementary information to restore the second frame.
  • the distributed video encoding apparatus for achieving the seventh object of the present invention includes obtaining a first frame encoder that encodes an input first frame, a packet loss rate (PLR), and a channel coding rate based on the obtained packet loss rate.
  • a channel monitor unit for determining a number of motion prediction steps based on the provided packet loss rate, and performing motion prediction according to the determined number of motion prediction steps and generating auxiliary information;
  • a second frame encoder configured to generate parity bits by encoding information and perform channel encoding according to the determined channel coding rate.
  • the channel monitor may determine twice the packet loss rate as a forward error correction (FEC) rate.
  • FEC forward error correction
  • the second frame encoder is a transform and quantizer for transforming and quantizing the generated auxiliary information, a first channel encoder for generating parity bits by channel coding the transformed and quantized data, and a first channel according to the channel coding rate.
  • a second channel encoder may perform channel encoding on the data provided from the encoder to compensate for channel loss.
  • the motion prediction unit may provide the distributed video decoding apparatus with the motion vector and the motion prediction performance step information used for the motion prediction.
  • the distributed video encoding apparatus for achieving the seventh object of the present invention described above determines a channel encoding rate and a size of video data to be transmitted based on a first frame encoder encoding a first frame, a packet loss rate, and an available bit rate.
  • a control unit for providing a control signal for adjusting video data size based on the determined size of the video data to be transmitted, a motion predictor for generating auxiliary information by performing motion prediction when a control signal is provided from the control unit;
  • the second frame encoder may generate a parity bit by encoding the auxiliary information and perform channel encoding according to the determined channel coding rate.
  • the controller determines at least one of the number of motion prediction steps and the quantization interval adjustment based on the determined size of video data to be transmitted as a method for adjusting video data size, and corresponds to a method for adjusting the determined video data size.
  • a control signal can be provided.
  • the motion prediction unit may perform motion prediction according to the number of motion prediction execution steps, which are control signals provided from the controller, and provide the distributed video decoding apparatus with motion vector and motion prediction execution step information used for the motion prediction.
  • the second frame encoder is provided from a first channel encoder and a first channel encoder according to the first channel encoder according to the transform and quantization unit, the first channel encoder to generate a parity bit by channel encoding the transformed and quantized data And a second channel encoder configured to perform channel encoding on the data to compensate for channel loss.
  • the transform and quantization unit may perform quantization according to the provided quantization interval information when quantization interval information is provided from the controller.
  • a method of encoding a plurality of distributed video encoding apparatuses including: performing distributed video encoding on a photographed image using the plurality of distributed video encoding apparatuses; And transmitting the image data on which the distributed video encoding has been performed to a distributed video decoding unit through a predetermined communication channel, wherein distributed video encoding performed on at least one of the plurality of distributed video encoding apparatuses comprises: a first frame and A distributed video encoding method for encoding an input image divided into a second frame, the method comprising: encoding the first frame and transmitting an encoded first frame; receiving a motion vector from a distributed video decoding apparatus; Corresponding to the second frame based on a vector Generating pair information, obtaining a prediction error of the auxiliary information based on the generated auxiliary information and the second frame, and quantizing the second frame based on the obtained prediction error.
  • the distributed video encoding method may include transforming the input second frame, generating the auxiliary information, transforming the generated auxiliary information, and after the second frame is quantized, The method may further include generating parity bits by performing channel encoding on the second frame. Obtaining the prediction error of the auxiliary information based on the generated auxiliary information and the second frame may obtain the prediction error based on the converted auxiliary information and the converted second frame.
  • at least one of the plurality of distributed video encoding apparatuses may be used as a depth image photographing apparatus for obtaining depth information of a subject, and the plurality of distributed video encoding apparatuses except the depth image photographing apparatus may include the subject.
  • the 3D may be photographed at a plurality of viewpoints, and a 3D image of the subject may be obtained based on the depth information provided from the depth image photographing apparatus and image information associated with the subject photographed at the plurality of viewpoints.
  • the plurality of distributed video encoding apparatuses may include a predetermined sensor for detecting whether a fire occurs in at least one of the plurality of distributed video encoding apparatuses, and the image data photographed by the plurality of distributed video encoding apparatuses and whether or not the fire is provided by the sensor.
  • the determination data can be transmitted using a wireless network.
  • the plurality of distributed video encoding apparatuses photograph individual images according to a plurality of photographing viewpoints, and the photographed individual images are decoded by a distributed video decoding method and provided as a multiview screen having a plurality of screens, or by a specific control signal. Only some of the captured images may be provided.
  • a distributed video decoding method when a monitored person appears in a surveillance region, motion is detected by motion sensors mounted in the plurality of distributed video encoding apparatuses, and the distributed video encoding apparatuses start to operate.
  • the compressed data can be transmitted in the wireless sensor network network, and the compressed data is transmitted and received to the router through the wireless sensor network network by the wireless communication module included in the plurality of distributed video encoding apparatuses.
  • the image data transmitted from the router to the gateway and transmitted to the gateway may be transmitted to the server through an Ethernet network.
  • the plurality of distributed video encoding apparatuses may acquire an infrared image of a predetermined subject by using at least one of the plurality of distributed video encoding apparatuses as an infrared image photographing apparatus, and may include an image captured by at least one of the plurality of distributed video encoding apparatuses. The image quality of the infrared image may be compared to detect whether the subject is forged.
  • an image encoding method using a plurality of image capturing apparatuses is distributed with respect to an image photographed using a distributed video encoder included in the plurality of image capturing apparatuses. Performing video encoding and transmitting the video data subjected to distributed video encoding to a distributed video decoding unit through a predetermined communication channel, wherein the distributed video encoding apparatus is performed in at least one of the plurality of distributed video encoding apparatuses.
  • the video encoding may include determining a channel condition, determining a channel coding rate and a size of video data to be transmitted based on the identified channel condition, adjusting the number of motion prediction steps based on the determined size of the video data to be transmitted, and quantization interval Adjust at least one method of video data
  • the method may include determining a method for controlling the data, performing encoding of video data to be transmitted according to the determined video data size adjusting method, and performing channel encoding on the encoded video data according to the determined channel coding rate. have.
  • the checking of the channel condition may obtain an available bit rate and a packet loss rate (PLR).
  • the determining of the number of motion prediction performing steps based on the determined size of the video data to be transmitted may determine the number of motion prediction performing steps among the total steps of TSS (Three Step Search) based on the determined size of the video data to be transmitted.
  • Encoding the video data to be transmitted by performing motion prediction corresponding to the determined number of motion prediction steps further includes providing the distributed video decoding apparatus with motion vector and motion prediction performance step information used for motion prediction. It may include.
  • the plurality of distributed video encoding apparatuses may include at least one of the plurality of distributed video encoding apparatuses as a depth image capturing apparatus for obtaining depth information of a subject, and the plurality of distributed video encoding apparatuses except the depth image capturing apparatus may detect the subject.
  • a 3D image of the subject may be obtained by photographing from a plurality of viewpoints, based on the depth information provided from the depth image capturing apparatus and image information related to the subject photographed from the plurality of viewpoints.
  • the plurality of distributed video encoding apparatuses may include a predetermined sensor for detecting whether a fire occurs in at least one of the plurality of distributed video encoding apparatuses, and the image data photographed by the plurality of distributed video encoding apparatuses and whether or not the fire is provided by the sensor.
  • the determination data can be transmitted using a wireless network.
  • the plurality of distributed video encoding apparatuses photograph individual images according to a plurality of photographing viewpoints, and the photographed individual images are decoded by a distributed video decoding method and provided as a multiview screen having a plurality of screens, or by a specific control signal. Only some of the captured images may be provided.
  • a distributed video decoding method when a monitored person appears in a surveillance region, motion is detected by motion sensors mounted in the plurality of distributed video encoding apparatuses, and the distributed video encoding apparatuses start to operate.
  • the compressed data can be transmitted in the wireless sensor network network, and the compressed data is transmitted and received to the router through the wireless sensor network network by the wireless communication module included in the plurality of distributed video encoding apparatuses.
  • the image data transmitted from the router to the gateway and transmitted to the gateway may be transmitted to the server through an Ethernet network.
  • the plurality of distributed video encoding apparatuses may acquire an infrared image of a predetermined subject by using at least one of the plurality of distributed video encoding apparatuses as an infrared image photographing apparatus, and may include an image captured by at least one of the plurality of distributed video encoding apparatuses. The image quality of the infrared image may be compared to detect whether the subject is forged.
  • the distributed video encoding / decoding apparatus As described above, according to the distributed video encoding / decoding method, the distributed video encoding / decoding apparatus, and the transform encoding apparatus according to the embodiment of the present invention, motion prediction between key pictures when generating auxiliary information used in the decoding process of distributed video encoding is performed.
  • the size of the block for variably is applied based on a predetermined criterion.
  • a motion vector generated by a distributed video decoding apparatus is provided to a distributed video encoding apparatus, and the distributed video encoding apparatus generates auxiliary information using the motion vector, and then determines a prediction error of the generated auxiliary information. To control quantization.
  • the size of video data to be encoded is reduced by adjusting the complexity and / or quantization interval of the encoding apparatus according to channel conditions, and data is added and transmitted to compensate for channel loss by the reduced data amount.
  • the performance of the reconstructed picture can be significantly improved.
  • the motion vector of the auxiliary information to be generated can obtain a more sophisticated motion vector than the motion vector estimated using the conventional fixed size.
  • the image quality of the reconstructed image can be improved by reducing the noise of the generated auxiliary information.
  • a conventional encoding apparatus may decode an image encoded by the distributed video encoding method through the transform encoding apparatus to which the above-described distributed video encoding and distributed video decoding methods are applied.
  • the loss robustness can be improved without occupying additional network resources, thereby reducing the probability of decryption failure.
  • quality of service (QoS) of delay-sensitive services can be guaranteed.
  • FIG. 1 is a block diagram illustrating a configuration of an encoder and a decoder corresponding thereto according to a conventional Weiner jib coding technique.
  • FIG. 2 is a conceptual diagram illustrating a conventional generation of auxiliary information using an interpolation method.
  • FIG. 3 is a conceptual diagram illustrating a conventional generation of auxiliary information using extrapolation.
  • FIG. 4 is a block diagram illustrating a configuration of an apparatus for decoding a distributed video encoded image according to an embodiment of the present invention.
  • FIG. 5 is a conceptual diagram illustrating a process of generating a motion vector according to an embodiment of the present invention.
  • FIG. 6 is a block diagram illustrating a configuration of a motion vector generator according to an embodiment of the present invention.
  • FIG. 7 is a flowchart illustrating a motion vector generation process according to an embodiment of the present invention.
  • FIG. 8 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • FIG. 9 is a flowchart illustrating a prediction error determination step and a quantization step in more detail according to another embodiment of the present invention.
  • FIG. 10 is a flowchart illustrating a distributed video decoding method according to another embodiment of the present invention.
  • FIG. 11 is a flowchart illustrating a configuration of a distributed video encoding apparatus and a distributed video decoding apparatus according to another embodiment of the present invention.
  • FIG. 12 is a block diagram showing a structure of a transform encoding apparatus according to another embodiment of the present invention.
  • FIG. 13 is a block diagram showing the structure of a transform encoding apparatus according to another embodiment of the present invention.
  • FIG. 14 is a block diagram illustrating a configuration of a transform encoder according to another embodiment of the present invention.
  • FIG. 15 is a block diagram showing a configuration of a transform encoding apparatus according to another embodiment of the present invention.
  • FIG. 16 is a block diagram illustrating a configuration in the case where the transform encoding apparatus illustrated in FIG. 15 has scalability on the time axis.
  • 17 is a graph illustrating a relationship between complexity and auxiliary information of an encoding apparatus.
  • FIG. 18 is a graph illustrating a change in performance according to a motion prediction ratio of an encoding apparatus.
  • 19 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • 20 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • 21 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • FIG. 22 is a block diagram illustrating a configuration of a distributed video encoding / decoding apparatus according to another embodiment of the present invention.
  • FIG. 23 is a block diagram illustrating a configuration of a distributed video encoding / decoding apparatus according to another embodiment of the present invention.
  • 24 is a conceptual diagram of a broadcast system using a plurality of video photographing apparatuses using distributed video coding according to an embodiment of the present invention.
  • 25 is a conceptual diagram illustrating a media sharing system using a plurality of video photographing apparatuses using a distributed video encoding method according to an embodiment of the present invention.
  • FIG. 26 is a conceptual diagram of a 3D object tracking apparatus using a distributed video encoding method, according to an embodiment of the present invention.
  • FIG. 27 is a block diagram illustrating a configuration of a multiview image processing apparatus according to an embodiment of the present invention.
  • FIG. 28 is a conceptual diagram illustrating a surveillance camera system using distributed video encoding according to an embodiment of the present invention.
  • 29 is a conceptual diagram of a monitoring system using a plurality of cameras according to an embodiment of the present invention.
  • FIG. 30 illustrates a forest fire detection and ignition tracking system employing a wireless imaging device according to an embodiment of the present invention.
  • FIG. 31 is a conceptual diagram illustrating a face detection system using a plurality of cameras according to an embodiment of the present invention.
  • 32 is a conceptual diagram illustrating a distributed video encoding apparatus including a plurality of distributed video encoders according to an embodiment of the present invention.
  • first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.
  • the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.
  • FIG. 4 is a block diagram illustrating a configuration of an apparatus for decoding a distributed video encoded image according to an embodiment of the present invention.
  • the distributed video encoding and decoding apparatus includes a distributed video encoding apparatus 400 for encoding a wine-jib frame and a distributed video decoding apparatus including a function of generating auxiliary information through motion prediction using a variable block. Contains 430
  • the distributed video encoding apparatus 400 includes a WZ frame encoder 410 and a keyframe encoder 420.
  • the WZ frame encoder 410 divides the frames of the source video content into wine-jib frames and key frames according to the wine-jib coding scheme. For example, even-numbered frames of source video content may be selected as key frames, or odd-numbered frames may be selected as wine-jib frames.
  • the WZ frame encoder 410 encodes the Weiner jib frames and the key frames and provides them to the distributed video decoding apparatus 430.
  • the distributed video decoding apparatus 430 includes a key frame decoder 460, a channel decoder 440, an image restorer 450, an auxiliary information generator 490, and a motion vector generator 480. ) And picture buffer 470.
  • the keyframe decoder 460 reconstructs the keyframe using the information received from the keyframe encoder 420.
  • the motion vector generator 480 determines the size of a block to be used for motion prediction using the reconstructed key frame.
  • the auxiliary information generator 490 generates auxiliary information for the current WZ frame to be reproduced by using the reconstructed key frame and the motion vector generated by the motion vector generator 480.
  • the channel decoder 440 estimates the quantized values by using the auxiliary information input from the auxiliary information generator 490 and the parity bits received from the distributed video encoding apparatus 400.
  • the channel decoder 440 determines that a reliable quantized value cannot be estimated when performing channel code decoding, the channel decoder 440 continuously distributes parity bits within a predetermined limit until reliable estimation is possible.
  • 400 may be configured to request and receive.
  • the distributed video encoding apparatus 400 receives only the parity amount necessary for decoding from the distributed video encoding apparatus 400, thereby being efficient in terms of rate-distortion performance. This is possible when there is a reverse channel (ie a feedback channel) requesting parity bits.
  • the channel decoder 440 may be configured to receive a predetermined amount of parity bits at once without a parity request each time, so as not to request parity on the reverse channel while using the same. have.
  • the distributed video decoding apparatus 430 may further configure to request the parity bits to the distributed video encoding apparatus 400. .
  • the distributed video encoding apparatus 400 sends a predetermined amount of parity calculated or set to the distributed video decoding apparatus 430, and the distributed video decoding apparatus 430 does not require parity bits. It may be configured in the form.
  • channel code used in the channel decoder 440 of FIG. 4 may use a turbo code that is found to have reached a Shannon limit or may use an LDPC channel code.
  • other channel codes with good coding efficiency and error correction can be used.
  • the image restorer 450 reconstructs the current WZ frame by using the quantized value and auxiliary information estimated by the channel decoder 440.
  • the auxiliary information is generated by the auxiliary information generator 490.
  • the auxiliary information generator 490 generally uses an interpolation method assuming linear change between frames in generating auxiliary information.
  • the change between video frames is caused by the movement of the object, the movement of the camera, the exposure area and the change of light. Except for changes in exposure area and light, the difference between these frames corresponds to the movement of pixels between frames. If the motion vector of the pixel is known correctly, it is possible to accurately predict the pixel of the frame to be interpolated.
  • the auxiliary information generator 490 generates auxiliary information based on the motion vector provided from the motion vector generator 480.
  • the auxiliary information generator 490 uses side information corresponding to the WZ frame based on the motion vector provided from the motion vector generator 480 using the keyframe reconstructed by the keyframe decoder 460. Generates and outputs auxiliary information to the channel decoder 440. The auxiliary information generator 490 generates linear information corresponding to the WZ frame to be reconstructed using interpolation, assuming linear motion between key frames located before and after the current WZ frame.
  • the motion vector generator 480 generates a motion vector based on the key frames and provides the generated motion vector to the auxiliary information generator 490.
  • the motion vector generator 480 estimates the motion of a rectangular section or block of a frame.
  • FIG. 5 is a conceptual diagram illustrating a process of generating a motion vector according to an embodiment of the present invention.
  • the motion vector generator 480 searches for another key frame 510 to find an MxN sample region 512 that matches the MxN sample block 502 of one key picture 500. Review area 520.
  • the motion vector generator 480 compares all or part of the MxN block 502 of one frame with the possible MxN blocks in the search area (generally, the area based on the position of the current block) 520, and the most of them. Find the matching region 512. Subsequently, the motion vector generator 480 generates a motion vector that is a difference value between the position of the current M ⁇ N block 502 and the position of the candidate region 512.
  • the block size for motion prediction may be one of 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4.
  • the motion vector generator 480 determines sizes of a plurality of macroblocks for motion prediction, obtains motion vectors using macroblocks having respective sizes, and selects an appropriate motion vector among the motion vectors. do.
  • FIG. 6 is a block diagram illustrating a configuration of a motion vector generator according to an embodiment of the present invention.
  • the moved vector generator 480 includes a block size selector 600, a motion prediction performer 610, a motion vector storage 620, and a motion vector determiner 630. It includes.
  • the block size determiner 600 determines the size of the macroblock for motion prediction based on the key frame.
  • the macroblocks may be square and / or rectangular and / or any shape in accordance with the present invention.
  • the block size determiner 600 determines a plurality of macroblock sizes.
  • the block size determining unit 600 may determine large block sizes for motion prediction when the motion is monotonous and there is no large difference between the key frames. If there is a lot of motion, small block sizes can be determined for motion prediction. Large block sizes include block sizes of 16x16, 16x8 and 8x16, and small block sizes may include 8x8, 8x4, 4x8 and 4x4 block sizes.
  • the block size determiner 600 may determine the sizes of all macroblocks that can be used for motion prediction for motion prediction. When the block size determiner 600 determines the block sizes for motion prediction, the block size determiner 600 provides the determined block sizes to the motion prediction performer 610.
  • the motion prediction performer 610 generates motion vectors by performing motion prediction on each of sizes of blocks used for motion prediction.
  • the motion prediction performing unit 610 searches an area of another key frame to find a sample area that matches the sample block of one key frame, finds the block that most matches the difference, and the difference therebetween. Create a motion vector as a value.
  • the motion prediction performer 610 repeats the process of generating and storing such a motion vector by the number of block sizes.
  • the motion prediction execution unit 610 stores the generated motion vector in the motion vector storage unit 620.
  • the motion vector selector 630 measures similarity between blocks by using motion vectors obtained with different block sizes present in a key frame, and selects a motion vector determined to be the most similar among them. That is, the motion vector selector 630 measures a similarity between the block of the one key frame and the corresponding block of the other key frame among the motion vectors and selects a motion vector having a block size corresponding to the highest similarity. Choose.
  • the similarity between each block present in the key frame is generally the sum of absolute difference (SAD) or mean absolute difference (MAD) of the blocks present in each key frame. It can be calculated using a method such as the average of the absolute sum of) or SSD (Sum of Square Difference, sum of squares of the difference of each block).
  • SAD absolute difference
  • MAD mean absolute difference
  • SSD SSD
  • the motion vector selector 630 provides the selected motion vector to the auxiliary information generator 490.
  • FIG. 7 is a flowchart illustrating a motion vector generation process according to an embodiment of the present invention.
  • the motion vector generator 480 first determines a plurality of macroblock sizes for motion prediction in step S710.
  • macroblock sizes may be determined according to characteristics of an image. For example, if there are fine or complex motions between frames or there is a lot of motion, small block sizes may be determined for motion prediction. Alternatively, when the motion between frames is monotonous and there is no big difference, large block sizes may be determined for motion prediction. Alternatively, the sizes of all macroblocks that can be used for motion prediction may be determined.
  • the motion vector generator 480 generates a motion vector by performing motion prediction according to the size of each macro block in step S720. Specifically, the area 520 of the other key frame 510 is searched to find the MxN sample area 512 that matches the MxN sample block 502 of one frame 500. The motion vector generator 480 compares all or part of the MxN block 502 of one picture with the possible MxN blocks in the search area (generally, the area based on the position of the current block) 520, and the most among them. The matching area, that is, the candidate area 512 is found. Subsequently, the motion vector generator 480 generates a motion vector that is a difference value between the position of the current M ⁇ N block 502 and the position of the candidate region 512.
  • the motion vector generator 480 determines whether a motion vector has been generated for the sizes of all macroblocks determined in operation S730. If a motion vector has been generated for all determined macroblock sizes, the motion vector generator 480 measures similarity between a block of one picture and a block of another frame for each motion vector in step S740.
  • the block may have a square shape.
  • a sum of absolute difference (SAD), a mean absolute difference (MAD), or a sum of square difference (SSD) may be used as a method of measuring similarity between blocks existing in a key frame.
  • SAD sum of absolute difference
  • MAD mean absolute difference
  • SSD sum of square difference
  • the scope of the present invention is not limited thereto, and the block size may be configured in any form other than square, and in the method of measuring similarity between blocks existing in a key frame, another arbitrary method may be selected.
  • the present invention can be applied in any case, such as the case of performing the auxiliary information by the interpolation method, the extrapolation method as shown in FIG.
  • the motion vector generator 480 selects a motion vector corresponding to the highest similarity among the generated motion vectors. That is, a motion vector having the largest similarity between blocks can ensure the best motion compensation.
  • the motion vector generator 480 outputs the motion vector selected in step S760 to the auxiliary information generator 490.
  • FIG. 8 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • the distributed video encoding apparatus first encodes an input key frame and transmits the encoded key frame to the distributed video decoding apparatus (step S810).
  • the distributed video encoding apparatus may encode the key frame through various known encoding techniques such as H.264 / AVC and MPEGx.
  • the distributed video encoding apparatus receives a motion vector from the distributed video decoding apparatus (step S820), performs motion compensation on a key frame encoded using the received motion vector, and generates auxiliary information corresponding to the WZ frame. (Step S830). Since the generated auxiliary information is the same as the auxiliary information generated by the distributed video decoding apparatus, the distributed video encoding apparatus may know the auxiliary information generated by the distributed video decoding apparatus.
  • the distributed video encoding apparatus transforms the generated auxiliary information (step S840).
  • the distributed video encoding apparatus converts the input original WZ frame (step S850), and determines a prediction error of the auxiliary information based on the converted original WZ frame and the auxiliary information converted in step S840 (step S860).
  • the conversion time point for the input original WZ frame is not limited to the order shown in FIG. 8, and may be executed corresponding to the time point at which the original WZ frame is input.
  • the distributed video encoding apparatus When the prediction error is determined, the distributed video encoding apparatus performs quantization based on the determined prediction error (step S870), and then performs parity bit generation by performing channel encoding on the quantized data to distribute the generated parity bits.
  • the distributed video encoding apparatus may perform quantization for each coding unit, that is, an image area having a predetermined size that is encoded together at one time, such as a macro block.
  • FIG. 9 is a flowchart illustrating a prediction error determination step and a quantization step in more detail according to another embodiment of the present invention.
  • the distributed video encoding apparatus in order to determine a prediction error of an auxiliary information frame, the amount of error (Count coeff.block ) generated in the auxiliary information frame, the position of the block in which the error occurs (M error ), and the crossing probability (Crossover) is calculated (step S860).
  • the calculated values are used to determine the quantization level of each block and whether to perform quantization.
  • a position M error in which an error occurs in the auxiliary information frame may be calculated using Equation 1 below.
  • Equation 1 ORI means an original WZ frame, and SI means an auxiliary information frame.
  • i is the index of the block and I is the number of blocks in the frame.
  • j is the index of the coefficient in the block, and J is the number of coefficients in the block.
  • k is an index of a bit plane, and when k is 0, it means the most significant bit, and as k increases, it means a lower bit.
  • K means the number of bit planes in the coefficient.
  • the amount of errors in the auxiliary information frame may be calculated using Equation 2 below.
  • encoding is performed by dividing the entire frame without encoding all at once.
  • Crossover probability may be measured using Equation 3 using the amount of error and the LDPCA code length in the range of the original WZ frame that is performed at one time.
  • the distributed video encoding apparatus determines the quantization level of the original WZ frame based on the calculated values (Ste S900). For example, the distributed video encoding apparatus may determine the quantization level by determining the quantization bit based on the calculated values.
  • the distributed video encoding apparatus determines whether an error occurs and an intersection probability is smaller than a preset threshold for each block in the WZ frame (steps S910 and S920), so that an error does not occur and the intersection probability is increased. Quantization is omitted for blocks smaller than the threshold value (step S930), and quantization is performed for blocks in which an error has occurred or crossover probability is greater than or equal to the threshold value (step S940).
  • the distributed video encoding apparatus performs channel encoding when quantization of all blocks in a frame is completed through steps S910 to S940.
  • the distributed video encoding method adaptively performs quantization for a block determined to be an error-free block by using M error indicating an error of a block. That is, if an error is concentrated in a block having an error, quantization is performed to lower the crossing probability, and if only the block having the error is encoded, if the crossing probability is low enough, the block without an error does not perform quantization. This allocates a large number of bits to an error-prone block, and allocates bits according to the crossing probabilities to improve the performance of the LDPCA, thereby enabling efficient use of the bits.
  • the distributed video encoding apparatus knows the channel decoding capability of the distributed video decoding apparatus, it is possible to decode the ratio of bits without errors and bits with errors in the encoding apparatus. Transmit according to the number of crossing probabilities. Therefore, although the amount of bits to be transmitted may increase according to the image, the probability of decoding failure is reduced as much as possible, thereby increasing the safety of the decoder and effectively using the bits.
  • FIG. 10 is a flowchart illustrating a distributed video decoding method according to another embodiment of the present invention.
  • a distributed video decoding apparatus decodes an encoded key frame transmitted from a distributed video encoder (step S1010).
  • the distributed video decoding apparatus may decode a key frame through various known decoding techniques such as H.264 / AVC and MPEGx.
  • the distributed video decoding apparatus generates a motion vector based on the decoded key frame (step S1020), and transmits the generated motion vector to the distributed video encoder (step S1030).
  • the distributed video decoding apparatus generates motion information by performing motion compensation using the generated motion vector (step S1040). For example, the distributed video decoding apparatus estimates a motion vector between reconstructed key frames and then obtains a motion vector of auxiliary information corresponding to a WZ frame to be reconstructed from the motion vectors between the reconstructed key frames in consideration of the distance between the frames. do.
  • the decoding unit in the key frame indicated by the motion vector of the obtained auxiliary information may be generated as auxiliary information.
  • the distributed video decoding apparatus corrects the noise included in the generated auxiliary information by using the parity bits transmitted from the distributed video encoding apparatus (step S1050), and then performs dequantization to perform a WZ frame. To be restored (step S1060).
  • FIG. 11 is a flowchart illustrating a configuration of a distributed video encoding apparatus and a distributed video decoding apparatus according to another embodiment of the present invention.
  • the distributed video encoding apparatus 1100 may include a keyframe encoder 1105, a buffer 1110, a motion compensator 1115, a first transform unit 1120, a second transform unit 1125, It may include a quantization control unit 1130 and a channel encoder 1140.
  • an input image to be encoded is classified into a key frame and a WZ frame so that the key frame is provided to the key frame encoder 1105 and the WZ frame is provided to the second converter 1125.
  • the key frame encoder 1105 After the key frame encoder 1105 encodes the key frame, the key frame encoder 1105 provides the encoded key frame to the motion compensator 1115 and the key frame decoder 1155 of the distributed video decoding apparatus.
  • the key frame encoder 1105 may encode the key frame through various known encoding techniques such as H.264 / AVC and MPEGx.
  • the buffer 1110 temporarily stores a motion vector (MV: Motion Vector) provided from the auxiliary information generator 1160 of the distributed video decoding apparatus 1150 and provides the buffer to the motion compensator 1115.
  • MV Motion Vector
  • the motion compensator 1115 reads a motion vector transmitted from the distributed video decoding apparatus 1150 and stored in the buffer 1110, and then performs motion compensation on a key frame encoded by using the read motion vector. After generating the information, the generated auxiliary information is provided to the first converter 1120.
  • the motion compensator 1115 since the motion compensator 1115 performs motion compensation on a key frame using the motion vector provided from the distributed video decoding apparatus 1150, the motion compensator 1115 generates the distributed video encoding apparatus 1100 and the distributed video decoding apparatus 1150.
  • the supplementary information is identical to each other, and thus, the distributed video encoding apparatus 1100 may recognize the supplementary information generated by the distributed video decoding apparatus 1150 and reflect the supplementary information in the encoding.
  • the first transform unit 1120 transforms the auxiliary information provided from the motion compensator 1115 to provide the quantization control unit 1130, and the second transform unit 1125 transforms the input original WZ frame to quantize it.
  • the control unit 1130 is provided.
  • the first and second converters 1120 and 1125 may use the same conversion method.
  • the quantization controller 1130 determines the prediction error of the auxiliary information based on the converted auxiliary information frame provided from the first transform unit 1120 and the original WZ frame provided from the second transform unit 1125, and then based on the determined prediction error.
  • the quantization level and quantization of each block constituting the original WZ frame are determined, and the quantization is performed based on the quantization level and then provided to the channel encoder 1140.
  • the channel encoder 1140 generates a parity bit for the quantized data using the channel code, and then converts the generated parity bit to the channel decoder 1170 of the distributed video decoding apparatus 1150. send.
  • the generated parity bit may be stored in a separate buffer (not shown) and then provided to the distributed video decoding apparatus according to a transmission request of the distributed video decoding apparatus.
  • the distributed video decoding apparatus 1150 may include a keyframe decoder 1155, an auxiliary information generator 1160, a transformer 1165, a channel decoder 1170, and an inverse quantizer 1175.
  • the keyframe decoder 1155 decodes the encoded key frame provided from the keyframe encoder 1105 of the distributed video encoding apparatus 1100 and provides the decoded keyframe to the auxiliary information generator 1160.
  • the key frame decoder 1155 may decode the key frame through various known decoding techniques such as H.264 / AVC and MPEGx.
  • the auxiliary information generator 1160 performs motion prediction using the decoded key frame and then performs motion compensation to generate auxiliary information corresponding to the WZ frame.
  • the auxiliary information generator 1160 provides the motion vector generated in the process of performing the motion prediction to the distributed video encoding apparatus 1100.
  • the auxiliary information generator 1160 estimates a motion vector between the reconstructed key frames, and then calculates a motion vector of the auxiliary information corresponding to the WZ frame to be restored from the motion vectors between the reconstructed key frames in consideration of the distance between the frames.
  • a decoding unit in a key frame indicated by the motion vector of the obtained auxiliary information may be generated as auxiliary information.
  • the converter 1165 converts the auxiliary information provided from the auxiliary information generator 1160 and provides the converted information to the channel decoder 1170, and the channel decoder 1170 is included in the converted data provided from the converter 1165.
  • the quantized data is estimated by correcting the noise by using parity bits transmitted from the distributed video encoding apparatus 1100.
  • the inverse quantizer 1175 receives the estimated quantization data from the channel decoder 1170 and inversely quantizes the WZ frame.
  • the ambiguity generated during inverse quantization is solved by referring to the auxiliary information provided from the auxiliary information generator 1160.
  • the distributed video decoding apparatus 1150 uses the motion vector used to predict auxiliary information corresponding to the WZ frame.
  • the auxiliary information is transmitted to the encoding apparatus 1100 and the distributed video encoding apparatus 1100 generates auxiliary information by performing motion compensation using the received motion vector, thereby providing the same auxiliary information as that generated by the distributed video decoding apparatus 1150.
  • the coding efficiency can be improved by calculating a prediction error between the generated auxiliary information and the original WZ frame and adjusting the quantization level and whether the quantization is based on the prediction error, thereby obtaining an image of excellent quality.
  • FIG. 12 is a block diagram showing a structure of a transform encoding apparatus according to another embodiment of the present invention.
  • FIG. 12 is a block diagram illustrating a structure of a transform encoding apparatus according to an embodiment of the present invention.
  • the distributed video encoding apparatus 1100 and the distributed video decoding apparatus 1150 illustrated in FIG. 11 are applied. Shown.
  • the distributed video decoding apparatus 1200 and the distributed video decoding apparatus 1260 included in the transform encoding apparatus 1250 illustrated in FIG. 12 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.
  • the trans-coder may decode an image encoded by the distributed video encoding apparatus 1200 according to an embodiment of the present invention in a decoding apparatus 1295 that does not perform conventional distributed video decoding. Perform the function of converting.
  • the transform encoding apparatus 1250 may include a distributed video decoding apparatus 1260 and an encoder 1290.
  • the distributed video decoding apparatus 1260 decodes an encoded key frame provided from the distributed video encoding apparatus 1200, generates auxiliary information using the decoded key frame, and distributes a motion vector generated in the auxiliary information generation process.
  • the video encoding apparatus 1200 is provided.
  • the distributed video decoding apparatus 1260 corrects auxiliary information generated by using the parity bits provided from the distributed video encoding apparatus 1200, and then dequantizes and restores the WZ frame.
  • the encoder 1290 encodes the received key frame and the WZ frame and then decodes the decoder ( 1295).
  • the encoder 310 may encode using various known coding techniques such as H.264 / AVC, MPEGx, for example.
  • the same standard technique as the decoding standard technique applied to the decoding apparatus is used. It can be applied to the encoding.
  • FIG. 13 is a block diagram showing the structure of a transform encoding apparatus according to another embodiment of the present invention.
  • the distributed video decoding apparatus 1300 illustrated in FIG. 13 and the distributed video decoding apparatus 1360 included in the transform encoding apparatus 1350 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.
  • the transform encoding apparatus 1350 may include a distributed video decoding apparatus 1360, a first encoder 1390, and a second encoder 1393.
  • the distributed video decoding apparatus 1360 decodes an encoded key frame provided from the distributed video encoding apparatus 1300, generates auxiliary information using the decoded key frame, and distributes a motion vector generated in the auxiliary information generation process.
  • the video encoding apparatus 1300 is provided.
  • the distributed video decoding apparatus 1360 corrects auxiliary information generated by using the parity bits provided from the distributed video encoding apparatus 1300, and then dequantizes and restores the WZ frame.
  • the WZ frame reconstructed by the distributed video decoding apparatus 1360 is provided to the first encoder 1390, and the reconstructed key frame is provided to the second encoder 1393.
  • the first encoder 1390 encodes the reconstructed WZ frame and provides it to the first decoding device 1396
  • the second encoder 1393 encodes the reconstructed key frame and provides the second decoding device 1399. do.
  • the first encoder 1390 and the second encoder 1393 may encode through various well-known encoding techniques such as H.264 / AVC, MPEGx, and not the distributed video encoding method.
  • the same standard technique as that applied to the corresponding decoding apparatus for providing the corrected data may be applied to the encoding.
  • FIG. 14 is a block diagram illustrating a configuration of a transform encoder according to another embodiment of the present invention.
  • FIG. 14 is a block diagram illustrating a structure of a transform encoder according to another embodiment of the present invention, in which a transform encoder 1450 to which the distributed video encoding apparatus 1400 and the distributed video decoding apparatus 1460 are applied is scalable in a time axis. For example, the case having scalablity is illustrated.
  • the distributed video decoding apparatus 1400 illustrated in FIG. 14 and the distributed video decoding apparatus 1460 included in the transform encoding apparatus 1450 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.
  • the transform encoding apparatus 1450 may largely include a base layer transform encoding unit 1455 and an enhancement layer transform encoding unit 1457.
  • the base layer transform encoder 1455 may include a decoder 1465 and a second encoder 1493
  • the enhancement layer transform encoder 1457 may include an auxiliary information generator 1470 and a converter 1475.
  • a channel decoder 1480, an inverse quantizer 1485, and a first encoder 1490 may be largely include a base layer transform encoding unit 1455 and an enhancement layer transform encoding unit 1457.
  • the base layer transform encoder 1455 may include a decoder 1465 and a second encoder 1493
  • the enhancement layer transform encoder 1457 may include an auxiliary information generator 1470 and a converter 1475.
  • a channel decoder 1480, an inverse quantizer 1485, and a first encoder 1490 may be included in the transform encoding unit 1455 and an enhancement layer transform encoding unit 1457.
  • the base layer transform encoder 1455 decodes an encoded key frame provided from the distributed video encoding apparatus 1400, and then re-codes the encoded key frame through the second encoder 1493 to provide it to the second decoding apparatus 1499. In addition, the base layer transform encoder 1455 provides the decrypted key frame to the enhancement layer transform encoder 1457.
  • the enhancement layer transform encoder 1157 generates auxiliary information based on the key frame provided from the base layer transform encoder 1455, and corrects the generated auxiliary information using the parity bits provided from the distributed video encoding apparatus 1400. After inverse quantization, the signal is encoded by the first encoder and then provided to the first decoder 1496. In this case, the enhancement layer transform encoder 1457 provides the distributed video encoding apparatus 1400 with the motion vector generated during the auxiliary information generation process.
  • the first encoder 1490 and the second encoder 1493 may encode through various known encoding techniques such as H.264 / AVC, MPEGx, and the like, rather than distributed video encoding. The same standard technology as that applied to the corresponding decoding apparatus for providing the same may be applied to the encoding.
  • the transform encoding apparatus encodes a key frame reconstructed by the base layer transform encoder 1455 and reconstructed by the enhancement layer transform encoder 1575.
  • the WZ frame is encoded and then provided to the second decoding unit 1499 and the first decoding unit 1496, respectively, so that the base layer outputs only the result decoded by the second decoding unit 1499, and the enhancement layer.
  • the resultant images decoded by the second decoding unit 1499 and the first decoding unit 1496 may be rearranged and output.
  • FIG. 15 is a block diagram showing a configuration of a transform encoding apparatus according to another embodiment of the present invention.
  • the distributed video encoding apparatus 1500 and the distributed video decoding apparatus 1560 included in the transform encoding apparatus 1550 illustrated in FIG. 15 perform the same functions as the components of the same code illustrated in FIG. 11 to avoid duplication. Detailed description will be omitted.
  • a key frame encoded by the distributed video encoding apparatus 1500 is provided to the key frame decoder 1565 of the transform encoding apparatus 1550 and simultaneously provided to the second decoding apparatus 1599. That is, the key frame decoded by the decoding unit 1565 of the transcoding apparatus 1550 is provided only to the auxiliary information generating unit 1570 to be used only for generating auxiliary information, and provides the decoded key frame to an external decoding device. I never do that. Therefore, the image decoded by the second decoding apparatus 1599 is an image encoded by the keyframe encoder 1505 of the distributed video encoding apparatus 1500, and the transform encoding apparatus 1550 decodes the encoded key frame by second decoding. No separate processing to provide to the device 1599 is performed.
  • the transform encoding apparatus 1550 corrects and inverse quantizes auxiliary information generated by using the parity bits provided from the distributed video encoding apparatus 1500, and then encodes the encoded information through the first encoder 1593 and then encodes the first decoding apparatus. Provided in (1596).
  • FIG. 16 is a block diagram illustrating a configuration in the case where the transform encoding apparatus illustrated in FIG. 15 has scalability on the time axis.
  • the transform encoding apparatus 1650 may largely include a base layer transform encoder 1655 and an enhancement layer transform encoder 1657.
  • the base layer transform encoder 1655 may include a decoder 1665.
  • the enhancement layer transform encoder 1657 may include an auxiliary information generator 1670, a converter 1675, a channel decoder 1680, and the like.
  • the dequantizer 1685 and the first encoder 1690 may be included.
  • the base layer transform encoder 1655 decodes the encoded key frame provided from the distributed video encoding apparatus 1600 and provides the decoded key frame to the auxiliary information generator 1670.
  • the key frame decrypted by the base layer transform encoder 1655 is provided only to the auxiliary information generator 1670 of the enhancement layer transform encoder 1657 and used only to generate the auxiliary information. It is not provided to an external decoding device.
  • the image decoded by the second decoding apparatus 1696 is an image encoded by the encoder 1605 of the distributed video encoding apparatus 1600, and the base layer transform encoder 1655 decodes the encoded key frame by second decoding. There is no separate processing to provide to the device 1696.
  • the enhancement layer transform encoder 1657 generates auxiliary information based on the key frame provided from the base layer transform encoder 1655, and corrects the generated auxiliary information using the parity bits provided from the distributed video encoding apparatus 1600. After inverse quantization, the signal is encoded by the first encoder 1690 and then provided to the first decoder 1693. In this case, the enhancement layer transform encoder 1657 provides the motion vector generated in the auxiliary information generation process to the distributed video encoding apparatus 1600.
  • the transform encoding apparatus 1650 encodes a key frame reconstructed by the base layer transform encoder 1655 and then transmits the encoded key frame to the enhancement layer transform encoder 1657. And the WZ frame reconstructed by the enhancement layer transform encoder 1657 and then provided to the first decoding device 1693 to output only the result decoded by the second decoding device 1696 in the case of the base layer.
  • the resultant images decoded by the second decoding device 1696 and the first decoding device 1693 may be rearranged and output.
  • 17 is a graph illustrating a relationship between complexity and auxiliary information of an encoding apparatus.
  • the horizontal axis represents a complexity ratio of the distributed video encoding apparatus, and indicates the complexity of motion prediction.
  • the vertical axis represents PSNR (Peak Signal to Noise Ratio) and indicates the quality of side information (SI).
  • the quality of the supplementary information is improved.
  • the quality of the auxiliary information is improved, the difference between the auxiliary information and the original image to be encoded is reduced, thereby reducing the parity bit required for decoding. Therefore, as the complexity of the distributed video encoding apparatus increases, the coding efficiency increases.
  • FIG. 18 is a graph illustrating a change in performance according to a motion prediction ratio of an encoding apparatus.
  • FIG. 18 is a graph illustrating a change in performance according to a motion prediction ratio of a distributed video encoding apparatus.
  • the ratio of motion prediction is adjusted using TSS (Three Step Search), which is one of motion prediction methods, and the corresponding PSNR is measured. Shown.
  • N denotes the total number of TSS steps
  • K denotes the number of TSS steps performed by the encoding apparatus.
  • coding efficiency improves as the size of K increases. For example, when PSNR is 30dB, K is 2, and the coding efficiency is about 20% higher than when K is 0. When K is 6, the coding efficiency is increased by about 35% compared to when K is 0. It was.
  • K 2
  • the packet loss rate PLR: Packet Loss
  • Up to 20% rate can be used to transmit coded data without quality degradation.
  • the encoding efficiency is increased by increasing the complexity of the encoding apparatus, thereby increasing the coding efficiency.
  • the loss robustness is increased by transmitting additional FEC data by the reduced bit amount according to the efficiency.
  • R a is defined as an available bit rate
  • r FEC is an FEC rate
  • a PLR is a packet loss rate
  • the decoding device may Only (1-PLR) R a is received normally.
  • the encoder determines the FEC rate (r FEC) using the predicted value of the available bit rate (R a) and the packet loss rate (PLR) that indicates the channel conditions.
  • the FEC rate r FEC is determined, the data transmitted by the encoding apparatus becomes (1-r FEC ⁇ PLR) ⁇ R a , and the FEC data becomes r FEC ⁇ PLR ⁇ R a .
  • the available bit rate (R a) and the packet loss rate (PLR) is also changed according to the channel conditions due to change in accordance with the channel status r FEC value.
  • the available bit-rate (R a) is reduced thereby reducing the size of the transmittable data FEC.
  • the encoding apparatus in order to minimize the increase in the use of network resources due to the addition of the FEC data, the encoding apparatus is increased to increase the coding efficiency, thereby reducing the size of data to be transmitted and reducing the FEC data by the amount of the reduced data. Adding increases loss robustness without additional network resources.
  • 19 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • 19 is a flowchart illustrating a distributed video encoding method according to an embodiment of the present invention, and illustrates a process of increasing loss robustness in consideration of both available bandwidth and loss according to channel conditions.
  • the encoder obtains the available bit rate (R a) and the packet loss rate (PLR) confirms the channel condition (Step S1910).
  • the encoding apparatus determines the FEC rate r FEC based on the obtained packet loss rate (step S1920).
  • the FEC rate may be determined to be twice the obtained packet loss rate (PLR), but is not limited thereto and may be adjusted to a different value according to channel conditions.
  • the encoding apparatus determines the size of the video data to be transmitted and the size of the FEC data based on the packet loss rate (PLR) and the FEC rate (step S1930).
  • the size of video data to be transmitted may be determined as (1-r FEC ⁇ PLR) ⁇ R a
  • the size of FEC data may be determined as r FEC ⁇ PLR ⁇ R a .
  • the encoding apparatus determines an encoding complexity for encoding according to the determined size of the video data (step S1940).
  • the determination of the coding complexity may be performed by determining the number of execution steps of a three step search (TSS), which is one of motion prediction methods, and the determination of the number of execution steps of the TSS may be performed based on a packet loss rate in units of a predetermined image frame. It may be determined in consideration of the number of TSS execution steps compared to PLR).
  • the number of TSS execution steps relative to the packet loss rate (PLR) may be stored in advance in the form of a lookup table, or may be obtained in real time by an arithmetic calculation.
  • the encoding apparatus performs encoding by performing motion prediction according to the encoding complexity determined in step S1940 (step S1950).
  • the encoding apparatus may provide the decoding apparatus with information such as a motion vector, a total number of TSS steps, and a number of TSS execution steps used in motion prediction.
  • the encoding apparatus performs channel encoding for compensating for channel loss, and then provides encoded data to the decoding apparatus (step S1960).
  • the encoding apparatus performs channel encoding according to the FEC rate determined in step S1920.
  • n video packets transmitted by the conventional encoding method are increased by increasing the complexity of the encoding apparatus according to channel conditions to improve encoding efficiency.
  • the robustness against loss can be improved without using additional network resources, thereby ensuring the quality of service.
  • 20 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • steps S2010 to S2030 shown in FIG. 20 perform the same functions as those of steps S1910 to S1930 shown in FIG. 19, detailed descriptions thereof will be omitted to avoid duplication.
  • the encoding apparatus adjusts the size of the video data to correspond to the determined video data size.
  • a method for determining is determined (step S2040).
  • a method for adjusting video size may include a method of adjusting a quantization interval, a method of increasing the complexity of an encoding apparatus, and a method of using the above two methods in combination. have.
  • the method of controlling the quantization interval has the advantage that there is no increase in complexity and the width of the adjustable data size is wide. However, as the size of the quantization interval increases, the quality of the video may be degraded. In addition, since the method of adjusting the complexity of the encoding apparatus performs motion prediction, the quality of the video can be kept the same, while the loss is large because the increase of the complexity limits the control of the video data size. There is a disadvantage that it is difficult to apply.
  • a method for adjusting the quantization interval in consideration of a preset priority for example, channel condition, performance of the encoding device, and / or user preference
  • the size of the video data can be adjusted by any one of a method of increasing the complexity of the method and a method of using the above two methods in combination.
  • the user can define the lowest quality reference value allowed by the user, and adjust the quality of the video by adjusting the quantization interval according to the channel situation. If the value falls below the defined reference value, the quantization interval adjustment can be stopped and the video data size can be adjusted through the complexity adjustment.
  • complexity control may be applied first, and when high priority is given to usage time, that is, power consumption, quantization interval adjustment may be applied to adjust the size of video data.
  • the encoding apparatus when the method for adjusting the size of the video data is determined in step S2040, the encoding apparatus performs encoding based on the determined method (step S2050), and performs encoding after performing channel encoding for compensating for channel loss.
  • the data is provided to the decoding apparatus (step S2060).
  • the encoding apparatus performs channel encoding according to the determined FEC rate.
  • 21 is a flowchart illustrating a distributed video encoding method according to another embodiment of the present invention.
  • FIG. 21 is a flowchart illustrating a distributed video encoding method according to another embodiment, and illustrates a decoding method when a distributed video encoding apparatus performs encoded motion to provide encoded data.
  • the decoding apparatus performs channel decoding on the encoded data transmitted from the encoding apparatus to restore an erasure according to channel loss (step S2110).
  • the erasure means data lost through channel loss.
  • the decoding apparatus then decodes the encoded key frame provided from the encoding apparatus (step S2120).
  • the decoding apparatus can decode the key frame through various known decoding techniques such as H.264 / AVC, MPEGx, etc., but the decoding apparatus decodes the key frame using the same standard as that used for encoding the key frame. It is preferable.
  • the decoding apparatus performs motion prediction using information such as a motion vector, a total number of TSS steps, and a number of TSS execution steps provided from the encoding device, to generate auxiliary information (SI) (step S2130).
  • information such as a motion vector, a total number of TSS steps, and a number of TSS execution steps provided from the encoding device, to generate auxiliary information (SI) (step S2130).
  • the decoding apparatus extracts the parity bits by decoding the data from which the channel loss is restored, corrects the generated auxiliary information using the extracted parity bits (step S2140), and inversely quantizes the corrected auxiliary information to restore the WZ frame. (Step S2150).
  • FIG. 22 is a block diagram illustrating a configuration of a distributed video encoding / decoding apparatus according to another embodiment of the present invention.
  • the distributed video encoding apparatus 2200 may include a keyframe encoder 2205, a channel monitor 2210, a motion predictor 2215, a transform and quantizer 2220. ), And may include a first channel encoder 2225 and a second channel encoder 2230, and encodes an input key frame to provide to a distributed video decoding apparatus, and moves a WZ frame according to a channel condition. After determining the number of prediction execution steps, motion prediction is performed according to the determined number of steps to generate parity bits, and the generated parity bits, motion vectors, and motion prediction performance information are provided to the distributed video decoding apparatus.
  • the keyframe encoder 2205 encodes the input keyframe and then provides the encoded keyframe to the motion predictor 2215 and the keyframe decoder 2270 of the distributed video decoding apparatus 2250.
  • the key frame encoder 2205 may encode the key frame through various known encoding techniques such as H.264 / AVC and MPEGx.
  • Channel monitor section 2210 is provided in the available bit rate (R a) and / or packet loss rate channel state information after confirming the channel status by obtaining the (PLR) (packet loss rate (PLR)) a motion predictor (2215). In addition, the channel monitor 2210 determines the FEC rate based on the packet loss rate (PLR), and then provides the determined FEC rate to the second channel encoder 2230.
  • PLR packet loss rate
  • the motion predictor 2215 determines the number of motion prediction execution steps (that is, the number of execution steps K of the TSS) based on the channel condition information (packet loss rate PLR) provided from the channel monitor unit 2210, and determines the determined motion prediction. Based on the number of steps performed, motion prediction is performed on the encoded key frame to generate auxiliary information, and the generated auxiliary information is provided to the transform and quantization unit 2220. In addition, the motion prediction unit 2215 may use the auxiliary information generation unit of the distributed video decoding apparatus 2270 to transmit information such as a motion vector (MV) used for motion prediction and the total number of TSS steps and the number of TSS steps that are motion prediction information. 2260).
  • MV motion vector
  • the transform and quantizer 2220 transforms and quantizes auxiliary information provided from the motion predictor 2215 and provides the transformed quantizer 2220 to the first channel encoder 2225.
  • the first channel encoder 2225 generates a parity bit for the quantized data using the channel code and provides the parity bit to the second channel encoder 2230.
  • the second channel encoder 2230 performs channel encoding to compensate for erasure due to channel loss in accordance with the FEC rate provided from the channel monitor unit 2210, and then distributes the data on which channel encoding is performed.
  • the decoding apparatus 2270 is provided.
  • the second channel encoder 2265 may perform channel encoding using, for example, a Reed-Solomon code.
  • the transformed and quantized unit may be used to convert the input WZ frame. 2220.
  • the distributed decoding apparatus 2250 may include a keyframe decoder 2255, an auxiliary information generator 2260, a second channel decoder 2265, and a first channel decoder 2270. And an image reconstructor 2275, which decodes an encoded keyframe provided from the distributed video decoding apparatus 2200, and performs motion prediction on the decoded keyframe using the provided motion vector and motion prediction information. After generating the auxiliary information, the WZ frame is recovered by removing noise included in the generated auxiliary information using the channel-decoded parity bit.
  • the keyframe decoder 2255 decodes an encoded keyframe provided from the keyframe encoder 2205 of the distributed video encoding apparatus 2200.
  • the auxiliary information generator 2260 generates the auxiliary information by performing motion prediction on the decoded key frame using information such as the motion vector, the TSS total number of steps, and the number of TSS execution steps provided from the distributed video encoding apparatus 2225. do.
  • the auxiliary information generated by the auxiliary information generator 2260 may be the same as the auxiliary information generated by the motion predictor 2215 since the auxiliary information is generated using the motion prediction information provided from the distributed video encoding apparatus 2200. have.
  • the second channel decoder 2265 channel-decodes the data provided from the distributed video encoding apparatus 2200 to reconstruct the eraser according to the channel loss.
  • the first channel decoder 2270 extracts a parity bit by performing channel decoding on the reconstructed data, and then estimates the quantized data by correcting the noise included in the auxiliary information using the extracted parity bit.
  • the image reconstructor 2255 reconstructs the WZ frame by inversely quantizing the data quantized estimated by the first channel decoder 2270 and performing inverse transformation. In this case, ambiguity generated during inverse quantization may be solved by referring to auxiliary information provided from the auxiliary information generator 2260.
  • FIG. 23 is a block diagram illustrating a configuration of a distributed video encoding / decoding apparatus according to another embodiment of the present invention.
  • a distributed video encoding apparatus 2300 may include a keyframe encoder 2305, a controller 2330, a motion predictor 2310, a transform and quantizer 2315, The first channel encoder 2320 and the second channel encoder 2325 may be included.
  • the keyframe encoder 2305 encodes the input keyframe and then provides the encoded keyframe to the motion predictor 2310 and the keyframe decoder 2355 of the distributed video decoding apparatus 2350.
  • the key frame encoder 2355 may encode the key frame through various well-known encoding techniques such as H.264 / AVC and MPEGx.
  • the controller 2330 may include a channel monitor 2333 and a resource manager 2336.
  • Channel monitor section 2333 is an available bit rate (R a) and the packet loss rate after confirming the channel status by obtaining the (PLR), obtained the available bit rate (R a) and the packet loss rate (PLR) the resource management unit (2336) to provide.
  • a resource management unit (2336) is FEC rate (r FEC) after setting the FEC rate (r FEC), is set on the basis of the available bit rate (R a) and the packet loss rate (PLR) as provided by the channel monitor section 2333 to the second The channel encoder 2325 is provided.
  • the resource manager 2336 may set the FEC rate r FEC to twice the packet loss rate PLR.
  • the resource management unit 2336 calculates the size of the video data ((1-r FEC ⁇ PLR) ⁇ R a ) and the size of the FEC data (r FEC ⁇ PLR ⁇ R a ), and calculates the calculated data size and a preset value.
  • the video data scaling method is determined based on the priority.
  • the resource manager 2336 may adjust the number of motion prediction performance steps (ie, the number of performance steps K of the TSS) or the quantization interval to correspond to the size of the calculated video data, and determine to perform the motion prediction and the quantization interval control in combination It may be. In this case, the resource manager 2336 adjusts the quantization interval in consideration of the channel condition, the performance of the encoding apparatus, and / or the user's preference, the motion prediction means step, and any one of a combination of the above two methods. After determining the method as a method of adjusting the size of the video data, a control signal corresponding to the determined may be provided.
  • the motion prediction unit 2310 performs motion prediction on the encoded key frame based on the motion prediction execution step number (that is, the execution step number K of the TSS), which is a control signal provided from the resource management unit 2336, and receives auxiliary information.
  • the generated auxiliary information is provided to the transform and quantization unit 2315.
  • the motion predictor 2310 may provide information such as a motion vector (MV) used for motion prediction and information such as the total number of TSS steps, the number of TSS steps, and the number of TSS execution steps, which are motion prediction information, to the auxiliary information generator of the distributed video decoding apparatus 2350 ( 2360).
  • MV motion vector
  • the transformer and quantizer 2315 transforms and quantizes auxiliary information provided from the motion predictor 2310 and provides the transformed quantizer 2315 to the first channel encoder 2320.
  • the transform and quantization unit 2315 may perform quantization corresponding to the quantization interval, which is a control signal provided from the resource manager 2336.
  • the first channel encoder 2320 generates a parity bit for the quantized data using the channel code and provides the parity bit to the second channel encoder 2325.
  • the second channel encoder 2325 performs channel encoding to compensate for erasure due to channel loss, and then provides the distributed video decoding apparatus 2350 with the data subjected to channel encoding.
  • the second channel encoder 2370 may perform channel encoding according to the FEC rate information provided from the resource manager 2336.
  • the distributed video decoding apparatus 2350 may include a keyframe decoder 2355, an auxiliary information generator 2360, a second channel decoder 2370, and a first channel decoder 2380. And an image restoration unit 2390.
  • the keyframe decoder 2355 decodes the encoded keyframe provided from the keyframe encoder 2305 of the distributed video encoding apparatus 2300.
  • the auxiliary information generator 2360 generates auxiliary information by performing motion prediction on the decoded key frame using information such as a motion vector, a TSS total number of steps, and a number of TSS execution steps provided from the distributed video encoding apparatus 2300. do.
  • the auxiliary information generated by the auxiliary information generator 2360 may be the same as the auxiliary information generated by the motion predictor 2310 since the auxiliary information generated by the auxiliary video encoding apparatus 2300 is generated using the motion prediction information provided from the distributed video encoding apparatus 2300. have.
  • the second channel decoder 2370 reconstructs the erasure according to the channel loss by channel decoding the data provided from the distributed video encoding apparatus 2300.
  • the first channel decoder 2380 extracts a parity bit by performing channel decoding on the reconstructed data of the eraser and then estimates the quantized data by correcting the noise included in the auxiliary information using the extracted parity bit.
  • the image reconstructor 2390 reconstructs the WZ frame by performing inverse quantization and inverse quantization of the quantized data estimated from the first channel decoder 2380.
  • the ambiguity generated during inverse quantization may be solved by referring to the auxiliary information provided from the auxiliary information generator 2360.
  • the above-described distributed video encoding method may be used in a plurality of image capturing apparatuses (distributed video encoding apparatuses), and data provided by the plurality of distributed video encoding apparatuses is decoded by the plurality of distributed video decoding apparatuses.
  • distributed video encoding apparatuses distributed video encoding apparatuses
  • data provided by the plurality of distributed video encoding apparatuses is decoded by the plurality of distributed video decoding apparatuses.
  • a plurality of distributed video encoding apparatuses and distributed video decoding apparatuses may be provided.
  • the distributed video encoding apparatus may be used in the same meaning as an image photographing apparatus and a camera.
  • an image capturing apparatus that is a distributed video encoding apparatus and a distributed video decoding apparatus that decodes data provided by the image capturing apparatus will not be described.
  • an image capturing apparatus or a camera according to an embodiment of the present invention will be described below.
  • the image capturing apparatus, the camera, and the distributed video encoding apparatus are used in the same sense.
  • the decoding apparatus for decoding the data transmitted from the image capturing apparatus or the camera apparatus the above-described distributed video encoding apparatus and distributed video are described.
  • the configuration of the decoding device can be used.
  • a component for capturing an image such as an image capturing apparatus or a camera, may include a distributed video encoding apparatus and an encoding method using distributed video coding according to an embodiment of the present invention.
  • the image capturing apparatus may include a WZ frame encoder and a key frame encoder to encode image information.
  • the image capturing apparatus may further include a buffer or a motion compensation unit in addition to the WZ frame encoder and the keyframe encoder to perform motion prediction using information provided from a distributed video decoder or a decoder.
  • the image capturing apparatus or the camera checks the state of the channel capable of transmitting data from the image capturing apparatus to the distributed video decoding apparatus, and determines the channel coding rate and the size of the video data to be transmitted based on the identified channel condition.
  • the channel monitor may be further included.
  • the distributed video decoding apparatus decodes a plurality of image data provided from an image capturing apparatus or a camera, and includes a keyframe decoder, a channel code decoder, an auxiliary information generator, and an image reconstruction unit, which are components included in the aforementioned distributed video decoding apparatus. It may include.
  • the distributed video decoding apparatus further includes an additional encoder to encode distributed video coded images in a general video decoding apparatus by encoding data decoded by the distributed video decoding apparatus using a general encoding method using a general encoding method rather than a distributed video encoding method. Enable decryption.
  • an apparatus for performing distributed video encoding will be described with an image capturing apparatus and a camera as an example.
  • a configuration in which the apparatus for performing distributed video encoding is provided outside the image capturing apparatus and the camera is also provided. It is included in the scope of right.
  • 24 is a conceptual diagram of a broadcast system using a plurality of video photographing apparatuses using distributed video coding according to an embodiment of the present invention.
  • a plurality of image capturing apparatuses 2400 may perform distributed video coding and images provided from a plurality of image capturing apparatuses 2400.
  • a metadata tagging unit 2410 for assigning attribute information to the distributed video decoding apparatus 2420 and a distributed video decoding apparatus 2420 for decoding information provided from the plurality of image capturing apparatuses 2400 may be included.
  • only a part of the plurality of image capturing apparatuses 2400 may perform distributed video coding, and other methods may use general video coding instead of distributed video coding. It is assumed that both the imaging apparatus 2400 perform distributed video coding on the images provided through the imaging units of the plurality of imaging apparatuses 2400 using distributed video coding.
  • the image capturing apparatus 2400 may include a distributed video encoding apparatus and encode an image using an encoding method using distributed video coding according to an embodiment of the present invention.
  • the image capturing apparatus 2400 may include a WZ frame encoder and a key frame encoder to encode image information.
  • the image capturing apparatus 2400 may further include a buffer or a motion compensator in addition to the WZ frame encoder and the key frame encoder to perform motion prediction using information provided from the distributed video decoder.
  • the image capturing apparatus 2400 checks the state of a channel capable of transmitting data from the image capturing apparatus to the distributed video decoding apparatus 2420, and thus the channel encoding rate and the video to be transmitted are determined based on the identified channel condition.
  • the channel monitor unit for determining the data size may be further included.
  • the metadata tagging unit 2410 is a part capable of assigning attribute information related to data to a plurality of image data photographed by the image capturing apparatus 2400, and provides location information of the image capturing apparatus 2400, information related to a photographing target, Various attribute information such as photographing object movement information and color characteristics of an image may be provided to the plurality of image data photographed by the image capturing apparatus 2400.
  • the metadata is data provided to a plurality of video data contents, and may selectively decode a plurality of videos generated based on metadata provided to the plurality of video data.
  • the metadata tagging unit 2410 is represented as an independent device for convenience of description, but when it is not included in the image capturing apparatus or it is not necessary to give attribute information to the image data, the metadata tagging unit 2410 ) May not be used.
  • the distributed video decoding apparatus 2420 is a portion that decodes a plurality of image data provided from the image capturing apparatus 2400, and includes a keyframe decoder, a channel code decoder, and an auxiliary component included in the distributed video decoding apparatus 2420. It may include an information generating unit and an image restoring unit.
  • the distributed video decoding apparatus 2420 includes an additional encoder to decode an image in a general video decoding apparatus by encoding data decoded by the distributed video decoding apparatus using a general encoding method using a general encoding method instead of a distributed video encoding method. To be able.
  • the plurality of image capturing apparatus 2400 photographs a playground in which a soccer game occurs. Some of the plurality of image capturing apparatuses 2400 photograph individual players, some of the entire stadium, and some of the plurality of image capturing videos of the goal situation from behind the goal post, each camera captures a video of various views according to the role.
  • the metadata tagging unit 2410 tags metadata to multiple video streams input from the plurality of image capturing apparatuses 2400.
  • the viewer may further include a component for selecting a view captured by a specific image capturing apparatus or a component for providing an image for selectively viewing a specific event based on tagged metadata. May selectively provide image data corresponding to a desired view to the user, or additionally provide a component that selectively provides only a specific event desired by the user.
  • 25 is a conceptual diagram illustrating a media sharing system using a plurality of video photographing apparatuses using a distributed video encoding method according to an embodiment of the present invention.
  • the media sharing system includes a media generating apparatus including a plurality of image capturing apparatuses 2500 and a media generating server 2510, and one or more clients 2520 connected to the media generating apparatus through a communication network.
  • Each of the image capturing apparatuses 2500 is provided with different unique identifiers (GUIDs), and photographs images of different viewpoints and provides them to the media generation server 2510.
  • the image capturing apparatus 2500 may be a camera for capturing an image.
  • At least one of the image capturing apparatuses 2500 may include a distributed video encoder to encode an image captured by the image capturer using distributed video encoding.
  • the image capturing apparatus 2500 may be connected to the media generation server 2510 by wire or wirelessly.
  • the image capturing apparatus 2500 and the media generation server 2510 may be connected through an IEEE 1394 network or a wireless network.
  • the media generation server 2510 may include a distributed video decoding apparatus and may decode image data transmitted from the image capturing apparatus 2500.
  • the distributed video decoding apparatus may be provided at the client 2520 side.
  • the media generation server 2510 allocates the same multicast address and different port numbers to each image photographed by the plurality of photographing apparatuses 2500, and includes a session description including an assigned multicast address and a port number. Protocol) message is generated and transmitted to the plurality of clients 2520. At this time, the media generation server 2510 receives images from the plurality of photographing apparatuses by executing one VP producer, and assigns the same multicast address and different port numbers to the plurality of received images.
  • the media generation server 2510 receives an SDP message and receives a multicast address different from the multicast address allocated through the SAP (Service Access Point) packet to the client 2520 through a communication network.
  • the SDP message includes service attribute information including at least one of an assigned multicast address and a plurality of port numbers, a producer's name, an e-mail address, a video codec, a video quality, and a service type.
  • the client 2520 analyzes the SDP message from the media generating device 2510 to extract service attribute information, and the image by the photographing apparatus 2500 selected by the user or the multi-view image by the plurality of photographing apparatuses 2500. Outputs
  • the client 2520 may display the multi-view image by the plurality of photographing apparatuses 2500 as a wide image using the network tiled display device.
  • 26 to 27 illustrate a configuration of an apparatus for photographing a stereoscopic image using distributed video encoding and decoding according to an embodiment of the present invention, which illustrates a specific embodiment of a system for photographing a stereoscopic image.
  • the stereoscopic image photographing system using the distributed video encoder or the distributed video decoder according to the above-described embodiment of the present invention may also be included in the scope of the present invention.
  • FIG. 26 is a conceptual diagram of a 3D object tracking apparatus using a distributed video encoding method, according to an embodiment of the present invention.
  • the three-dimensional object tracking device is a frame of an image acquired at the same time from at least two (preferably three or more) video cameras (multi-view camera) 2600 At least one depth map acquired at the same time from a multi-view video storage unit 2610 storing data and at least two (preferably three or more) depth cameras 2605.
  • depth map storage unit 2620 for storing map data, camera information including a camera focal length of each viewpoint, a camera correction unit 2630 for obtaining a base matrix of positions and directions between viewpoints, and camera information
  • An object region extracting unit 2640 for extracting a moving object region from the background in the image using a matrix matrix, a multiview image of the previous frame and the current frame, and a depth map data;
  • the object motion estimation unit 2650 for detecting a motion object of the body region and obtaining three-dimensional motion vectors of the object, and detecting a shielding region in the object region, and then performing the backward motion estimation of the previous frame and the current frame.
  • a shielding area processor 2660 for estimating the position of the obscured object.
  • the apparatus may further include a multiview image corrector 2635 for correcting the epipolar line of the image acquired by the multiview camera 2600 to coincide using the base matrix.
  • the apparatus further includes an object motion display unit 2670 for displaying the extracted object motion on the screen.
  • the apparatus may further include a multiview camera control and driver 2680 for adjusting the position of the multiview camera 2600 such that the object is located at the center region in the image in case the object deviates from the image.
  • the multiview camera 2600 may encode the image photographed through the image pickup unit through distributed video encoding using the distributed video encoder included in the multiview camera 2600.
  • Distributed video encoding performed by the multiview camera 2600 may use the same method as the above-described method.
  • the multi-view video storage unit 2610 and the depth map storage unit 2620 store video images obtained from several video cameras and one depth camera in a data storage device such as a video tape or a frame capture device.
  • the stored image information is used as an input by the camera corrector 2630, the multiview image corrector 2635, the object region extractor 2640, and the object motion estimator 2650.
  • the multi-view video storage unit 2610 and the depth map storage unit 2620 may be connected to a plurality of video cameras and one depth camera by a wired or wireless network.
  • the distributed video decoder may be provided at a portion of the multiview video storage unit 2610 to process the image, and may decode the distributed video encoded image generated by the multiview camera 2600.
  • the camera corrector 2630 extracts feature points between various images that know the distance from the camera 2600, and then obtains a rotation matrix and a movement matrix for mutual conversion using the correspondence between the feature points and the actual distance information. .
  • camera information such as focal length is obtained through a camera correction technique. This information is used to continue object tracking by using a camera at another point in time when the object motion estimation unit 2650 does not generate the shielding area when the shielding area is generated.
  • the multi-view image corrector 2635 uses the camera transformation matrix (base matrix) obtained by the camera corrector 2630 to refer to the epipolar line of the image acquired by the remaining cameras with respect to the reference camera. Make it parallel to the rows of the image. This allows you to reduce the search area and time for object tracking and stereo matching.
  • the object region extractor 2640 extracts the pixels of the object region based on a criterion of the object and the background pixel using the change in brightness and depth of the sequentially input current image and the previously input image, and includes all of the pixels.
  • a bounding box is placed on the current or next frame, and the coordinates of the start and end points are stored in the buffer. This is used to find the motion vector in the frame following the object motion.
  • the object motion estimator 2650 compares the object region of the current frame extracted by the object region extractor 2640 with the object region of the next frame, and then compares the object region of the next frame with the smaller row and column size in the next frame. By selecting and moving the points in the detected block area toward the center of the block, the position of the block having the highest similarity is detected. At this time, the motion vector between the center points of each block is detected as a motion vector in the x and y directions. At this time, when the similarity is smaller than the predetermined threshold or when the matching is performed backward, the center of the center point is regarded as that there is no shielding area in the block only when the position difference from the original block center is within a certain number of pixels. The motion vector is obtained, and if not, the corresponding area is detected by the shielding area processor 2660 as the shielding area.
  • FIG. 27 is a block diagram illustrating a configuration of a multiview image processing apparatus according to an embodiment of the present invention.
  • a multiview image processing apparatus includes a multiview camera 2700, a depth camera 2710, a three-dimensional warping unit 2720, a trimap generator 2730, an alphamat generator 2740, A foreground extractor 2750 is included.
  • the multi-view camera 2700 is composed of a plurality of cameras (first to n-th cameras, 2700-1, 2700-2, and 2700-n) as shown.
  • the viewpoints of the plurality of cameras are different from each other according to the position of the camera, and thus a plurality of images having different viewpoints are bundled together to be called multiview images.
  • the multi-view image obtained from the multi-view camera 2700 includes color information for each pixel of the 2D image forming each image, but does not include depth information of the 3D image.
  • the depth camera 2710 acquires a depth map having depth information in three dimensions.
  • the depth camera 2710 is a device capable of illuminating a laser or an infrared ray to an object or a target area, obtaining a return beam, and obtaining depth information in real time.
  • the depth camera 2710 includes a depth sensor that senses depth information through a laser or infrared ray.
  • At least one of the multiview camera 2700 and the depth camera 2710 may include a distributed video encoding apparatus. Images captured by the image capture unit using the distributed video encoding apparatus included in the multiview camera 2700 and the depth camera 2710 may be encoded, and the encoded video data may be a trimap generator 2730 or a 3D warping unit 2720. ) And may be decoded by a distributed video decoding apparatus that may be included in the ramimap generator 2730 or the 3D warping unit 2720 or converted into another image format.
  • Image data provided by the multiview camera 2700 and the depth camera 2710 may be provided to the trimap generator 2730 or the 3D warping unit 2720 using wired or wireless.
  • the multiview image obtained from the multiview camera 2700 and the multiview depth map from the 3D warping unit 2720 are input to the trimap generator 2730, and the trimap generator 2730 is provided.
  • the multiview trimap from the trimap generator 2730 is input to the alphamat generator 2740, and the alphamat generator 2740 uses the input multiview trimap to provide a multiview alpha corresponding to the multiview image. You can create a matte.
  • the multiview alphamat generated from the alphamat generator 2740 is input to the foreground extractor 2750, and the foreground extractor 2750 applies a multiview alphamat to the multiview image from the multiview camera 2700. Multiview foreground can be extracted.
  • the trimap generator 2730 uses a target image and a three-dimensional warped depth map to generate a trimap corresponding to each image forming a multiview image.
  • the trimap generator 2730 may include a segment part, a foreground area determiner, a hole remover, and an erosion / expansion calculator.
  • the segment unit generates segmented images by performing color segmentation on each image forming a multiview image.
  • Color segmentation refers to a process of dividing an image into regions (segments) of similar colors.
  • the segment unit sets segments between pixels whose luminance or chrominance difference between adjacent pixels is less than or equal to a threshold in the target image to which color segmentation is to be performed.
  • the depth value does not exist at all or is very sparse in an area far from the depth camera 2710 according to the setting of the depth camera 2710, and the depth camera 2710 In the area nearer to), all depth values exist or are very dense.
  • all depth values exist or are very dense.
  • the three-dimensional warped depth map except that a hole, which is a pixel whose depth value is not defined, is generated due to the three-dimensional warping.
  • the discontinuity point of the depth value may occur at the boundary of the segment, it may be determined that the region having a large depth value among the segments is the foreground region, and the other region is the background region.
  • the foreground area determiner segments each of the three-dimensional warped depth maps corresponding to the segmented image by the segment unit, and sets the foreground area of the set of segments having a depth value greater than or equal to a certain ratio in the segment.
  • Decide on The ratio may be arbitrarily determined appropriately, for example, to determine the foreground area as a set of segments having a depth value of 70% or more.
  • the erosion / expansion calculator generates an immersion trimap by applying erosion and expansion operations to the input multiview depth map.
  • the trimap is the image before obtaining the alpha matte, which is essential for matting, divided into the foreground and the background, and the unknown region whether it is the foreground or the background.
  • the erosion / expansion calculating unit generates each image of the depth map constituting the input multiview depth map as a binary image according to a predefined threshold value, and then erodes the edge of the binary image inward and extends it out to generate an unknown region. This creates a multiview trimap.
  • the hole removing unit fills the holes with appropriate depth values in the foreground area of the three-dimensional warped depth map using surrounding pixel values. Parts outside the foreground area are no longer of interest, so you can only fill the hall with the foreground area.
  • FIG. 28 is a conceptual diagram illustrating a surveillance camera system using distributed video encoding according to an embodiment of the present invention.
  • the motion is detected by the motion sensor mounted on the low power wireless camera and the camera is immediately operated.
  • the information captured by the camera is compressed to be transmitted in the wireless sensor network, and the compressed data is transmitted to the gateway 2820 through the wireless sensor network.
  • the image data transmitted to the gateway 2820 is transmitted to the server 2830 through an Ethernet network, and the image data transmitted to the server 2830 is viewed through the monitoring system 2840.
  • the low power wireless camera 2800 Since the low power wireless camera 2800 operates in the existing sensor network, the low power wireless camera 2800 must be operated by a battery in the same manner as the operation of the sensor nodes in the existing sensor network system. Therefore, it is based on low power operation due to the limitation of power supply capacity. However, since the video is basically based on continuous shooting, the motion monitoring sensor is mounted on the camera 2800 to minimize the power consumption by operating the camera 2800 only when the motion of the monitoring area is detected.
  • the camera 2800 includes a distributed video encoding apparatus capable of performing distributed video encoding, and the distributed video encoding apparatus may encode an image captured by the camera through distributed video encoding.
  • the low power wireless camera 2800 may include a battery, a micro controller unit (MCU), a motion detection sensor, a distributed video encoding apparatus, and a wireless communication module.
  • MCU micro controller unit
  • the motion detection sensor may be installed at the front of the wireless camera 2800 to detect a motion of a person in an area to be monitored. At this time, the detected signal is notified to the MCU.
  • the camera 2800 functions to capture an image of the surveillance area.
  • the MCU applies power to the camera 2800 to start recording. If no movement is detected for a certain period of time, shooting ends.
  • an image captured by the wireless camera 2800 may be encoded through distributed video encoding.
  • Wireless signals are compressed to less than or equal to 1M bandwidth.
  • the MCU controls the battery and the wireless communication module, receives the motion detection from the motion detection sensor, and determines the operation of each part and applies power.
  • the wireless communication module wirelessly transmits distributed video encoded images in a distributed video encoding apparatus and is in charge of communication with a wireless sensor network.
  • the battery supplies power to each part under the control of the MCU.
  • the same scheme as described above can also be used in, for example, a baby monitoring system for monitoring a home using a wireless camera that encodes an image captured using distributed video coding indoors.
  • 29 is a conceptual diagram of a monitoring system using a plurality of cameras according to an embodiment of the present invention.
  • a surveillance system using a plurality of cameras 2900 may form three-dimensional data using a plurality of cameras 2900.
  • a surveillance system using a plurality of cameras 2900 is connected to a plurality of cameras 2900 and a plurality of cameras 2900, which are installed in a plurality of monitored areas, from a camera 2900.
  • a central monitoring unit 2910 is configured to receive captured image data and to control an operation of each camera 2900.
  • the plurality of cameras 2900 may encode an image photographed through the imaging unit through distributed video encoding using a distributed video encoder included in the camera 2900.
  • Distributed video encoding performed by the camera 2900 may use the same method as described above.
  • the central monitoring apparatus 2910 of the present invention includes a GIS database 2920 for displaying geographic information of a region to be monitored as 3D image data, and an actual image screen captured by the camera 2900 and a GIS database 2920.
  • a GIS database 2920 for displaying geographic information of a region to be monitored as 3D image data
  • an actual image screen captured by the camera 2900 and a GIS database 2920.
  • a plurality of cameras may be linked to photograph a specific point or a monitoring object.
  • the method of acquiring 3D coordinates of a camera image using 3D spatial data constructs 3D spatial data for providing a 3D image of an area to be monitored. Calculate the change of angle of view according to the zoom change of the camera, synchronize the screen of the camera with the screen of the 3D image, calculate the 3D coordinates for a specific point of the actual camera screen, and then use the 3D coordinates of the camera image using the 3D spatial data.
  • the 3D coordinates of the image may be obtained by using a method of obtaining.
  • FIG. 30 illustrates a forest fire detection and ignition tracking system employing a wireless imaging device according to an embodiment of the present invention.
  • the forest fire detection can be used as an example using a wireless image pickup device including a sensor other than the sensor for detecting the forest fire.
  • the plurality of wireless image capturing apparatuses 3000 are installed in place of loading in a wide area such as a mountain or a national park for detecting a fire and tracking a fire. Since the wireless image capturing apparatus 3000 according to the present invention is a wireless device, it is easier to install in a mountainous region than a general image camera or CCTV which should be connected by wire.
  • the wireless image capturing apparatus 3000 may include a distributed video encoding apparatus capable of performing distributed video encoding, and may encode an image captured by using distributed video encoding according to an embodiment of the present invention.
  • Image data encoded through distributed video encoding may be provided to a distributed video decoder to be decoded.
  • the image data generated by the wireless image capturing apparatus 3000 is collected by the gateway 3010 and passed through a router 3020 to a surveillance or control center in charge of forest fire detection and ignition tracking via a communication modem 3030 through a communication network. Can be sent.
  • the control center can accurately know the point of imaging. Therefore, using the unique number already assigned to each wireless image capturing apparatus 3000 and the installed point matching table, it is possible to clearly grasp the state of the point reflected by the wireless image capturing apparatus 3000. When appearing in the imaging device 3000 it is also possible to accurately determine the location of the missing person.
  • the control server (not shown) of the control center may determine the occurrence of fire through the change of the image while periodically receiving the image generated by the wireless image pickup device 3000.
  • the fire detector may detect the fire from the CCTV and the thermal imaging camera, but may determine the fire from the detection signal of the fire detection sensor 3040. Alternatively, the fire detector may be located at a control center to determine the fire.
  • a plurality of fire detection sensors 3040 may be employed to facilitate the occurrence of fire.
  • the fire detection sensor 3040 may use a sensor that detects temperature, smoke, or flame, but is not necessarily limited thereto. Any sensor capable of detecting a fire may be used.
  • the sensor may also use a fire detection sensor that senses one or more combinations of temperature, smoke, or flame.
  • the fire sensor 3040 may be installed separately from the wireless image capturing apparatus 3000 or may be installed in combination with the wireless image capturing apparatus 3000. Fire detection sensing signal can be transmitted to the control server of the control center, and in addition to the location of the fire detection sensor 3040 will be easier to determine in which position the fire occurred.
  • the control server of the control center can determine the location of the fire, and if the location is detected, the position of the surrounding wireless video imaging device 3000 to capture the location of the fire. Can be fixed selectively. By doing so, it is possible to secure more imaging data of the fire occurrence point.
  • FIG. 31 is a conceptual diagram illustrating a face detection system using a plurality of cameras according to an embodiment of the present invention.
  • FIG. 31 is a block diagram illustrating a fake face detection apparatus using an infrared image according to an exemplary embodiment of the present invention.
  • the image acquirer 3110, the face region extractor 3120, the fake face detector 3130, and the face recognizer are shown. 3140.
  • the image acquisition unit 3110 includes an infrared image acquisition unit 3113 and a general image acquisition unit 3116.
  • the infrared image acquisition unit 3113 acquires an infrared image captured by the infrared camera 3103 and provides the infrared image to the forged face detection unit 3130.
  • the general image obtaining unit 3116 obtains a general image captured by the general camera 3106 and provides the same to the face region extracting unit 3120.
  • the infrared camera 3103 and the general camera 3106 may be provided with a distributed video encoder, and distributed video encoding may be performed using the distributed video encoder.
  • the distributed video encoded data by the infrared camera 3103 and the general camera 3106 may be wirelessly transmitted to a specific server, and the server may determine whether the user is based on the transmitted data.
  • the face region extractor 3120 is a component-based technique using an image processing technique with respect to a general image input from the general image acquirer 3116, or a pattern recognition technique of Adaboost or SVM (Support Vector Machine, SVM). Extracts a face region and provides it to the forged face detector 3130.
  • Adaboost or SVM (Support Vector Machine, SVM).
  • the fake face detector 3130 includes a face image evaluator, a face / no face discrimination unit, and an eyeball characteristic analyzer for evaluating the image quality characteristics of the infrared image.
  • the face image evaluator evaluates the image quality of the infrared image extracted by the face region extractor 3120 and coincides with the input face region by using the technique of Sharpness, and as a result of the evaluation, the sharpness of the infrared image is lower than that of the face region. It detects with a fake face and provides a detection result of the fake face to the face recognition unit 3140. On the other hand, if the face region and the infrared image coincide with each other, the fake face is detected as not a fake face. The detection result is provided to the face recognition unit 3140.
  • the face presence / non-determination unit is based on the characteristics of the infrared image having the characteristic that the shape of the face is not visualized in the case of the infrared image of the photograph or the fake face, and the face region extracted by the face region extraction unit 3120 is input.
  • Adaboost or SVM to discriminate the presence or absence of a face, if the face is absent, it is detected as a fake face, and the detection result of the fake face is provided to the face recognition unit 3140, If the face is unique, it is detected as not a fake face, and a detection result of not being a fake face is provided to the face recognizing unit 3140.
  • the eyeball characterization unit detects and presets an eyeball area in the actual infrared image of the user in advance, and then analyzes between the predetermined eyeball areas and the eyeball area of the infrared image input from the infrared image acquisition unit 3113 and analyzes the eyeball area.
  • the detected face is detected as a fake face, and the detection result of the fake face is provided to the face recognition unit 3140. If any of the eyeballs and the eyeballs of the infrared image are the same, it is detected as not a fake face, and the detection result of not detecting the fake face is provided to the face recognizing unit 3140.
  • the face recognition unit 3140 performs face recognition on the face region input from the face region extraction unit 3120 according to the evaluation result, the determination result, and the analysis result of not being a fake face input from the fake face detection unit 3130. On the other hand, the face recognition unit 3140 does not perform face recognition according to the evaluation result and the determination result and the analysis result of the forgery face input from the fake face detection unit 3130, or the infrared camera 3103 and the general camera 3106. Control to retry shooting.
  • the present invention captures a face image by using an infrared / visible light camera at the same time, and analyzes the characteristics of the dual input infrared image to detect whether or not a fake face, and in the case of non-falsified face, the image captured by the visible light
  • 24 to 31 illustrate some embodiments in which distributed video encoding according to an embodiment of the present invention is used, and when distributed video encoding according to the above-described embodiment of the present invention can be used.
  • the present invention is not limited to the above-described embodiment and may be utilized in various forms.
  • 32 is a conceptual diagram illustrating a distributed video encoding apparatus including a plurality of distributed video encoders according to an embodiment of the present invention.
  • the distributed video encoding apparatus 3200 may include a plurality of distributed video encoders 3200-1, 3200-2, and 3200-n, and the distributed video decoding apparatus may include a plurality of distributed video decoders ( 3210).
  • each of the distributed video encoders 3200-1, 3200-2, and 3200-n and the distributed video decoder 3210 are shown as separate components, but a plurality of distributed video encoders 3200 and a plurality of distributed units are shown.
  • the video decoder 3210 may have an integrated configuration that shares some configurations.
  • each encoder is represented by an encoder that performs distributed video encoding.
  • the encoder may have a plurality of distributed video encoders and may use a general encoder that does not perform distributed video encoding.
  • Each distributed video encoder 3200 and distributed video decoder 3210 may have a configuration of the distributed video encoder 3200 described above.
  • the distributed video encoder 3200 may include a keyframe encoder, a quantizer, a block unitizer, and a channel code encoder having the above-described configuration.
  • the distributed video encoder 3200 may include a motion compensator, a first converter, and a quantization controller. , A second transform unit, a motion predictor, a channel monitor, and the like.
  • the distributed video decoder 3210 may include a keyframe decoder, an auxiliary information generator, a channel code decoder, and an image reconstructor, and may further include a picture buffer, a motion vector generator, a first encoder, It may include a second encoder.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Detection And Prevention Of Errors In Transmission (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention porte sur un procédé de codage/décodage vidéo distribué et sur un appareil de codage/décodage vidéo distribué qui peuvent améliorer la tolérance aux pertes et la qualité de service. Le procédé de codage vidéo distribué consiste d'abord à vérifier l'état d'un canal, à déterminer un rendement de codage de canal et la taille de données vidéo devant être transmises sur la base de l'état vérifié d'un canal, à déterminer le nombre d'étapes d'exécution de prédiction de mouvement sur la base de la taille déterminée des données vidéo devant être transmises, à coder les données vidéo devant être transmises par l'exécution de prédictions de mouvement conformément au nombre déterminé d'étapes d'exécution de prédiction de mouvement, et à soumettre les données vidéo codées à un codage de canal conformément au rendement de codage de canal déterminé. Par conséquent, il est possible d'améliorer la tolérance aux pertes, même sans occuper de ressources réseau supplémentaires, ce qui permet de réduire la probabilité d'échec de décodage.
PCT/KR2011/000194 2010-01-11 2011-01-11 Procédé et appareil de codage/décodage vidéo distribué et appareil de transcodage WO2011084037A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/521,552 US9210436B2 (en) 2010-01-11 2011-01-11 Distributed video coding/decoding method, distributed video coding/decoding apparatus, and transcoding apparatus

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
KR10-2010-0002080 2010-01-11
KR1020100002080A KR101074919B1 (ko) 2009-01-09 2010-01-11 분산 비디오 부호화 영상의 복호화 장치 및 그 방법
KR20100002926 2010-01-13
KR10-2010-0002926 2010-01-13
KR1020100012743A KR101059318B1 (ko) 2010-01-13 2010-02-11 분산 비디오 부호화/복호화 방법, 분산 비디오 부호화/복호화 장치 및 변환 부호화 장치
KR10-2010-0012743 2010-02-11
KR1020100026598A KR101103974B1 (ko) 2010-03-25 2010-03-25 분산 비디오 부호화/복호화 방법 및 분산 비디오 부호화/복호화 장치
KR10-2010-0026598 2010-03-25

Publications (2)

Publication Number Publication Date
WO2011084037A2 true WO2011084037A2 (fr) 2011-07-14
WO2011084037A3 WO2011084037A3 (fr) 2011-11-10

Family

ID=44305994

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2011/000194 WO2011084037A2 (fr) 2010-01-11 2011-01-11 Procédé et appareil de codage/décodage vidéo distribué et appareil de transcodage

Country Status (1)

Country Link
WO (1) WO2011084037A2 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016099116A1 (fr) * 2014-12-17 2016-06-23 상명대학교 서울산학협력단 Appareil et procédé permettant de fournir une radiodiffusion personnalisée orientée objet et support de stockage de métadonnées associées
CN114466206A (zh) * 2022-01-04 2022-05-10 北京快乐茄信息技术有限公司 直播数据处理方法及装置、电子设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR920020461A (ko) * 1991-04-06 1992-11-21 이헌조 영상신호의 움직임 벡터 검출방식 및 그 장치
KR100728031B1 (ko) * 2006-01-23 2007-06-14 삼성전자주식회사 가변 블록 크기 움직임 예측을 위한 부호화 모드 결정 방법및 장치
KR20070084686A (ko) * 2006-02-21 2007-08-27 (주)씨앤에스 테크놀로지 Ｈ.264 부호화를 위한 움직임 추정에서의 고속 다중 참조프레임 선택 방법

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR920020461A (ko) * 1991-04-06 1992-11-21 이헌조 영상신호의 움직임 벡터 검출방식 및 그 장치
KR100728031B1 (ko) * 2006-01-23 2007-06-14 삼성전자주식회사 가변 블록 크기 움직임 예측을 위한 부호화 모드 결정 방법및 장치
KR20070084686A (ko) * 2006-02-21 2007-08-27 (주)씨앤에스 테크놀로지 Ｈ.264 부호화를 위한 움직임 추정에서의 고속 다중 참조프레임 선택 방법

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016099116A1 (fr) * 2014-12-17 2016-06-23 상명대학교 서울산학협력단 Appareil et procédé permettant de fournir une radiodiffusion personnalisée orientée objet et support de stockage de métadonnées associées
CN114466206A (zh) * 2022-01-04 2022-05-10 北京快乐茄信息技术有限公司 直播数据处理方法及装置、电子设备及存储介质
CN114466206B (zh) * 2022-01-04 2024-03-15 北京快乐茄信息技术有限公司 直播数据处理方法及装置、电子设备及存储介质

Also Published As

Publication number Publication date
WO2011084037A3 (fr) 2011-11-10

Similar Documents

Publication Publication Date Title
WO2018048078A1 (fr) Procédé de codage/décodage d'image multivue synchronisée à l'aide d'informations de structure spatiale et appareil associé
WO2015137783A1 (fr) Procédé et dispositif de configuration d'une liste de candidats de fusion pour le codage et le décodage de vidéo intercouche
WO2020180155A1 (fr) Procédé et appareil de traitement de signal vidéo
WO2015152608A4 (fr) Procédé de décodage vidéo intercouche conçu pour réaliser une prédiction basée sur un sous-bloc, appareil associé, procédé de codage vidéo intercouche conçu pour réaliser une prédiction basée sur un sous-bloc, et appareil associé
WO2021177652A1 (fr) Procédé et dispositif de codage/décodage d'image permettant d'effectuer une quantification/déquantification de caractéristiques et support d'enregistrement permettant de stocker un flux binaire
WO2014109594A1 (fr) Procédé et dispositif pour coder une vidéo entre couches pour compenser une différence de luminance, procédé et dispositif pour décoder une vidéo
WO2020055161A1 (fr) Procédé et appareil de décodage d'image basés sur une prédiction de mouvement dans une unité de sous-blocs dans un système de codage d'image
WO2016117930A1 (fr) Procédé et appareil de décodage de vidéo inter-couche, et procédé et appareil de codage de vidéo inter-couche
WO2020055107A1 (fr) Procédé et appareil de décodage d'image basé sur une prédiction de mouvement affine à l'aide d'une liste de candidats mvp affine dans un système de codage d'image
WO2019027286A1 (fr) Procédé et appareil pour traiter un signal vidéo à l'aide d'une prédiction affine
WO2020180129A1 (fr) Procédé et dispositif de traitement de signal vidéo destiné à une prédiction inter
WO2015093920A1 (fr) Procédé de codage vidéo inter-couches utilisant une compensation de luminosité et dispositif associé, et procédé de décodage vidéo et dispositif associé
WO2020184964A1 (fr) Procédé et appareil de traitement de signal vidéo pour prédiction inter
WO2021172881A1 (fr) Procédé et appareil de codage/décodage d'image par prédiction inter, et support d'enregistrement stockant un train de bits
WO2021006579A1 (fr) Procédé et dispositif de codage/décodage de vidéo permettant de dériver un indice de poids pour une prédiction bidirectionnelle de candidat de fusion et procédé de transmission de flux binaire
WO2020184952A1 (fr) Procédé et dispositif de traitement de signal vidéo permettant de traiter des informations de différence de vecteur de mouvement pour l'interprédiction dans un signal vidéo
WO2020251269A1 (fr) Procédé de décodage d'image et dispositif associé
WO2020204418A1 (fr) Procédé de codage de vidéo ou d'image et dispositif associé
WO2020050695A1 (fr) Procédé et appareil de décodage d'images basé sur une prédiction de mouvement utilisant une liste de candidats de fusion dans un système de codage d'images
WO2015137789A1 (fr) Procédé de codage vidéo multicouche et procédé de décodage vidéo multicouche à l'aide d'un bloc de profondeur
WO2015137780A1 (fr) Procédé de codage vidéo multicouche et procédé de décodage vidéo multicouche utilisant des informations de motif
WO2015005754A1 (fr) Procédé et appareil de décodage inter-couches dune vidéo utilisant une prédiction de synthèse de vues, et procédé et appareil de codage inter-couches d'une vidéo utilisant une prédiction de synthèse de vues
WO2014163453A1 (fr) Procédé et appareil d'encodage vidéo intercouche et procédé et appareil de décodage vidéo intercouche permettant de compenser une différence de luminance
WO2021162446A1 (fr) Procédé et appareil de diffusion en continu d'une image de vr
WO2021060834A1 (fr) Procédé et dispositif de codage/décodage d'image basé sur une sous-image, et procédé de transmission de flux binaire

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11731975

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 13521552

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 11731975

Country of ref document: EP

Kind code of ref document: A2