WO2016108318A1

WO2016108318A1 - Method, device and system for synthesizing image in error-resistant compression region

Info

Publication number: WO2016108318A1
Application number: PCT/KR2014/013138
Authority: WO
Inventors: 최윤식; 홍순기; 강호약간저릭
Original assignee: 연세대학교 산학협력단
Priority date: 2014-12-31
Filing date: 2014-12-31
Publication date: 2016-07-07

Abstract

Disclosed are a method, device and system for synthesizing an image. The device for synthesizing an image of the present invention comprises: a stream syntax analysis unit receiving bitstreams of a plurality of input images, and performing syntax analysis on each of the received plurality of bitstreams; a parameter and header generation unit modifying the syntaxes analyzed in the stream syntax analysis unit according to configuration information of a pre-set synthesized image, thereby generating modified syntaxes; a macro block encoding syntax modification unit re-encoding the most probable mode (MPM) of each of a target macro block included in each of the plurality of bitstreams, and re-encoding entropy encoding syntaxes of the target macro blocks, thereby re-encoding the encoding syntaxes of the target macro blocks; and a stream synthesis unit replacing the current syntaxes included in the bitstreams by inserting the modified syntaxes generated in the parameter and header generation unit and the syntaxes re-encoded in the macro block encoding syntax modification unit, and outputting a bitstream of the synthesized image comprising the plurality of input images.

Description

Image Synthesis Method, Apparatus and System for Error Robust Compression Region

The present invention relates to a method, apparatus and system for image synthesis, and more particularly, to a method, apparatus and system for image synthesis in an error-tolerant compressed region capable of resolving highest probability mode mismatch in inter-screen encoding mode during image synthesis. .

International standardization organizations for video signal compression include the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) formed for video applications such as video conferencing, and ISO / IEC for various applications such as storage or broadcasting of video data. . ITU-T and ISO / IEC together form JVT (Joint Video Team) to improve compression performance over existing Moving Picture Experts Group (MPEG) -2, H.263, and MPEG-4 Visual video compression coding standards. 264 / AVC video coding standard was established. H.264 / AVC is currently being evaluated as the next generation video compression technology. In particular, it is widely used in video multimedia services such as multi-channel high-definition video compression, Internet, cable modem, video transmission, and digital data broadcasting by combining with next generation multimedia services such as digital TV, satellite and terrestrial digital multimedia broadcasting (DMB). It is used.

The H.264 / AVC-based fast image synthesis technique is a technique for generating a single synthesized image by combining a plurality of independent images. One of the conventional high speed image synthesis techniques is disclosed in Korean Patent Registration No. 10-1053161 (published on January 28, 2011). The disclosed feature of the prior art is to synthesize a plurality of images into a single image and then generate a bitstream for transmission through an encoding process, and to receive a plurality of bitstreams for each of the already encoded and generated images. After extracting and modifying only a specific portion of each of the plurality of received bitstreams, a bitstream of one synthesized image is generated.

That is, the above-described prior art, which discloses a video synthesis technique in the H.264 / AVC compression region, does not perform decoding on a plurality of input image bitstreams. Instead, syntax analysis is performed on a sequence parameter set (SPS), a picture parameter set (PPS), and a slice header included in each of the plurality of input image bitstreams. The sequence parameter set (SPS), the picture parameter set (PPS), and a part of the slice header of each of the plurality of input image bitstreams are changed or newly set to correspond to the display form of the composite image. That is, the conventional technology does not decode a plurality of input image bitstreams applied in a compressed and compressed state, but performs syntax analysis, parameter change, and variable change of a plurality of input image bitstreams in a compressed region in an encoded state to synthesize a composite image. Generates and outputs a bitstream in H.264 / AVC format for.

Accordingly, there is an advantage in that the synthesis process of the plurality of input image bitstreams and the encoding process of the composite image bitstream can be greatly reduced to generate a composite image at low complexity and high speed.

1 is a diagram illustrating a problem of a conventional image synthesis technique.

In FIG. 1, (a) shows one of a plurality of input images before generating a composite image, and (b) shows a composite image including the input image of (a). As shown in (a) of FIG. 1, each of the plurality of input images before synthesis does not have other images around them. Therefore, no other macro block exists outside the boundary of the input image. However, in the composite image shown in (b) of FIG. 1, the input image of (a) is disposed at the lower right side of the composite image. As the input image of (a) is disposed at the lower right side of the synthesized image, other input images that did not exist before being synthesized are disposed adjacent to the left boundary and the upper boundary of the input image of (a) so that the macro block exists. do.

2 shows nine prediction modes for intra prediction encoding.

In H.264 / AVC, nine prediction modes are provided in intra prediction encoding for improving compression ratio, and one of the modes is selected to generate and encode a prediction block. In FIG. 2, the number assigned to each prediction mode is a number assigned according to the probability of occurrence for each prediction mode, and the lower the number, the higher the probability of occurrence. The image encoding apparatus encodes and transmits prediction mode information together with a prediction block, and requires 4 bits to transmit nine prediction modes. However, H.264 / AVC additionally provides Most Probable Mode (MPM) to reduce the number of bits for transmitting prediction mode information. The MPM determines the highest probability mode by referring to the prediction mode of blocks arranged in the periphery during intra prediction encoding. When the prediction mode and the highest probability mode are the same, only one bit of a flag bit for expressing the same reduces the number of data bits to be transmitted. If the prediction mode is not the same as the optimal mode, the flag bit indicates that the prediction mode and the highest probability mode are not the same, and additional 3 bits are added to the information of one of the eight prediction modes except the prediction mode selected in the highest probability mode. Express That is, if the prediction mode and the highest probability mode are the same, only one bit is used to reduce the number of data bits to be transmitted, and only four bits are required even if the prediction mode and the highest probability mode are not the same. Has

MPM, which is currently defined in H.264 / AVC, provides A prediction mode that refers to blocks arranged adjacent to the left and B prediction mode that refers to blocks arranged adjacent to the top. The lower number prediction mode is selected as the MPM among the mode numbers. Referring to the conventional image synthesis scheme of FIG. 1 in consideration of the MPM, in (a), since only one input image exists alone, the average mode (DC) of the prediction mode for the macroblock in the upper left is prediction mode 2 in FIG. It is set to). MPM is therefore also set to DC.

On the other hand, in (b), since the other images are arranged adjacent to the left and the top of the input image, the A prediction mode of the MPM is the vertical mode (VER) which is the prediction mode 0 according to the prediction mode of the left macroblock, and the B prediction mode. Is a horizontal mode (HOR) that is prediction mode 1 according to the prediction mode of the upper macroblock. Since A prediction mode is prediction mode number 0 and B prediction mode is prediction mode number 1, A prediction mode with a lower mode number is selected as MPM, so that MPM is set to prediction mode 0 which is vertical mode (VER). Since this is different from the DC mode which is the MPM in a single image, the MPM mismatch problem for the corresponding macroblock in the synthesized image occurs.

The discrepancy of the MPM generates an unintended motion vector value at the time of decoding the macro block, which may cause deterioration of the image quality or, in some cases, the syntax information exceeds the set limit range, thereby causing the decoder to stop operating. It becomes a factor.

This MPM mismatch problem is a problem that occurs because the prior art did not sufficiently consider the macro block data when generating the synthesized image, and must be corrected to generate an error-free composite image.

In addition, in a conventional image synthesis technique, when a macro block is encoded in a skip mode, an error that is encoded in a skip mode may occur even when the macro block is not encoded in the skip mode due to the skip mode encoding syntax of the macro blocks of other images to be merged. There was this.

In addition, in the conventional image synthesis technique, there is a problem in that an error of incorrectly selecting a code table used for entropy encoding occurs due to the influence of other images to be combined in entropy encoding.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image synthesizing apparatus which generates a composite image at low complexity and high speed without generating an error due to a mismatch between the MPM mismatch and the composite image in skip mode encoding and entropy encoding.

Another object of the present invention is to provide an image synthesis system for achieving the above object.

It is another object of the present invention to provide an image synthesis method for achieving the above object.

According to an embodiment of the present invention, an image synthesizing apparatus receives bitstreams of a plurality of input images, and performs entropy decoding up to at least a macroblock layer for each of the plurality of received bitstreams. A stream parser for parsing; A parameter and header generator configured to modify the syntax analyzed by the stream parser according to configuration information about a preset composite image to generate a modified syntax; Means for regenerating and re-coding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or performing entropy encoding on a transform coefficient encoded for the target macroblock. Re-encoding the encoded syntax of the target macroblock using any one or all of the means for re-encoding the entropy encoded syntax including information on the number of non-zero transform coefficients. Macroblock encoding syntax correction; And inserting the modified syntax generated by the parameter and header generation unit and the syntax re-encoded by the macroblock encoding syntax corrector by replacing the existing syntax included in the bitstream, and including the plurality of input images. A stream synthesizer for outputting a bitstream of the synthesized video; It includes.

The macroblock encoding syntax correcting unit is arranged to be adjacent to each of the plurality of macroblocks and a plurality of boundary macroblocks, which are macroblocks arranged on a boundary of the input video, among the target macroblocks, from each of the plurality of bitstreams. Decodes and extracts a neighboring macroblock that is the macroblock of another input image, analyzes the extracted boundary macroblock and the prediction mode set in each of the neighboring macroblocks, and analyzes the MPM (Most Probable) according to the prediction mode of the neighboring macroblock. An MPM correction unit for re-encoding the encoding syntax of the boundary macroblock by regenerating a mode; And an entropy encoding syntax recoding unit configured to reencode the encoding syntax of the target macroblock by modifying the entropy encoding syntax including information on the number of non-zero transform coefficients in the entropy encoding for the target macroblock. It characterized by including.

The macroblock encoding syntax corrector further includes a macroblock padding unit configured to pad an additional macroblock to one side of the target macroblock and to encode the padded macroblock to generate an encoding syntax for the padded macroblock. The synthesis unit further inserts an encoding syntax for the padded macro block encoded by the macro block padding unit, and outputs a bitstream of the synthesized image including the plurality of input images.

The entropy encoding syntax re-encoding unit re-encodes the boundary macroblock among the target macroblocks and performs the entropy encoding using the variable length coding (VLC) method for the boundary macroblock. Modifying the VLC table for the boundary macro block using the neighboring macro block of another input image arranged adjacent to the macro block, and using the modified VLC table, the non-zero of the transform coefficients of the boundary macro block. The variable length encoding syntax including information about the number of transform coefficients is modified to re-encode the encoding syntax of the boundary macroblock.

The entropy encoding syntax re-encoding unit modifies a skip mode encoding syntax of the boundary macroblock, wherein the macroblocks included in the other input image arranged adjacent to the image including the boundary macroblock are encoded in skip mode. The apparatus may further include a skip mode encoding syntax correcting unit configured to correct an encoding syntax regarding a skip mode of the boundary macroblock.

The MPM corrector determines a boundary of an image according to an arrangement position in which the plurality of input images are arranged in the composite image, and a boundary block extractor that decodes and extracts the boundary macro block and the neighboring macro block from the determined boundary surface. ; A prediction mode analyzer configured to analyze and determine the prediction mode set in each of the boundary macro block and the neighboring macro block; MPM reproduction which determines the MPM prediction mode of the boundary macroblock by using the prediction mode set in the neighboring macroblock, regenerates the MPM by determining whether the prediction mode of the boundary macroblock and the recalculated MPM prediction mode are the same. part; And a prediction mode recoding unit encoding the reproduced MPM to be inserted into a bitstream of the synthesized image. Characterized in that it comprises a.

The boundary block extractor may include an image arrangement analyzer configured to determine an arrangement position of each of the plurality of input images in the synthesized image based on configuration information of the synthesized image set in the parameter and the header generator; An image boundary discrimination unit which determines an interface between adjacent images generated according to a position where a plurality of input images are arranged in the composite image; A boundary macroblock extracting unit configured to decode and extract the boundary macroblocks disposed on an interface between the images in a current image for correcting an MPM among the plurality of input images; And a neighboring macroblock extraction unit configured to decode and extract the neighboring macroblock disposed adjacent to the boundary macroblock from images disposed adjacent to the current image. Characterized in that it comprises a.

The prediction mode analyzer includes: a current block prediction mode analyzer configured to analyze and determine the prediction mode set in the boundary macro block; And a neighboring block prediction mode analyzer configured to analyze and determine the prediction mode set in the neighboring macroblock. Characterized in that it comprises a.

The MPM regenerator determines the MPM prediction mode by determining a prediction mode having the lowest assigned number among the prediction modes of each of the neighboring macroblocks, and if the prediction mode of the MPM prediction mode and the boundary macroblock is the same, a flag If only the bit is set and the prediction mode of the boundary macroblock is different from the MPM prediction mode, additional bits for designating the prediction mode of the boundary macroblock together with the flag bits are set.

The MPM regenerator may select the prediction mode having the lowest number among the prediction modes of the neighboring macroblocks disposed adjacent to the left and the top of the boundary macroblock as the MPM prediction mode.

The parameter and header generation unit may generate a modified syntax by modifying headers of parameters and slices of the SPS and the PPS among the syntaxes of the plurality of bit streams.

The parameter and header generator may be configured to receive configuration information about the composite image from the outside.

According to an aspect of the present invention, there is provided a video synthesis system, comprising: a plurality of encoding devices configured to receive different original images, encode a predetermined encoding scheme, and output a bitstream; And receiving a plurality of input images as the plurality of bitstreams, correcting the syntax for each of the plurality of bitstreams according to configuration information about a preset composite image, and generating a modified syntax to generate the existing syntax included in the bitstream. By inserting the phrases alternately, a bitstream of the composite image is generated, and each of the plurality of boundary macroblocks is a macroblock disposed at a boundary of the input image from each of the plurality of bitstreams. Decode and extract neighboring macroblocks which are the macroblocks of other adjacently arranged input images, analyze the extracted boundary macroblock and the prediction mode set in each of the neighboring macroblocks, according to the prediction mode of the neighboring macroblocks. Regenerate and recode MPM (Most Probable Mode) An image synthesizing apparatus for inserting a bitstream of the synthesized image; The apparatus for synthesizing a video may include: a stream syntax analyzer configured to receive and parse the plurality of bitstreams; A parameter and header generator for modifying the syntax analyzed by the stream parser to generate the modified syntax; Means for regenerating and re-coding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or performing entropy encoding on a transform coefficient encoded for the target macroblock. Re-encoding the encoded syntax of the target macroblock using any one or all of the means for re-encoding the entropy encoded syntax including information on the number of non-zero transform coefficients. Macroblock encoding syntax correction; And inserting the modified syntax generated by the parameter and header generation unit and the syntax re-encoded by the macroblock encoding syntax corrector by replacing the existing syntax included in the bitstream into the bitstream for the synthesized image. A stream synthesizer configured to output a bitstream of the synthesized image including a plurality of input images; Characterized in that it comprises a.

According to an aspect of the present invention, there is provided a video synthesizing method including a stream parser, a parameter and header generator, a macroblock encoding syntax corrector, and a stream synthesizer. A synthesis method, comprising: receiving, by a stream parser, a plurality of bitstreams for each of a plurality of input images and parsing each of the received bitstreams; Generating a modified syntax by modifying the syntax analyzed by the parameter and the header generator according to configuration information about a preset composite image; Means for regenerating and re-encoding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or a transform coefficient encoded for the target macroblock. Means for re-encoding the entropy encoding syntax including information on the number of non-zero transform coefficients in performing entropy encoding on the target macroblock by using any one or all of the means. Macroblock encoding syntax modification step of re-encoding encoding syntax; And inserting the modified syntax generated by the parameter and header generation unit and the syntax re-encoded by the macroblock encoding syntax corrector by replacing the existing syntax included in the bitstream by replacing the existing syntax included in the bitstream. Outputting a bitstream for a composite image including; Characterized in that it comprises a.

Accordingly, the image synthesizing method, apparatus and system of the present invention does not decode and encode a plurality of input image bitstreams, but changes the parameters and variables in the encoded compression state to generate the macroblock encoded at low complexity high speed composite image generation. It is possible to eliminate errors or deterioration of the synthesized image which may be caused by the prediction mode mismatch of the highest probability mode.

2 shows nine prediction modes for intra prediction encoding.

3 is a block diagram schematically illustrating an image synthesizing system according to an embodiment of the present invention.

4 shows the configuration of the image synthesizing apparatus of FIG.

FIG. 5 shows a detailed configuration of the macroblock encoding syntax correction unit of FIG.

FIG. 6 shows a detailed configuration of the MPM corrector of FIG.

7 shows an image synthesis method according to an embodiment of the present invention.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings which illustrate preferred embodiments of the present invention and the contents described in the accompanying drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In addition, in order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

Throughout the specification, when a part is said to "include" a certain component, it means that it may further include other components, without excluding the other components unless otherwise stated. In addition, the terms "... unit", "... unit", "module", "block", etc. described in the specification mean a unit for processing at least one function or operation, which means hardware, software, or hardware. And software.

Referring to FIG. 3, the image synthesizing system of the present invention includes a plurality of image input means V1 to Vn, a plurality of encoding apparatuses D1 to Dn, and an image synthesizing apparatus VC.

The plurality of image input means V1 to Vn are implemented by a camera to generate an original image and transmit the original image to corresponding encoding devices D1 to Dn among the plurality of encoding devices D1 to Dn.

The plurality of encoding apparatuses D1 to Dn respectively receive original images from corresponding image input means among the plurality of image input means V1 to Vn, encode them according to H.264 / AVC format, and encode the encoded bitstream BS1. ~ Bn) is transmitted to the image synthesizing apparatus VC. In FIG. 3, for the convenience of description, the number of the image input means V1 to Vn and the number of the encoding devices D1 to Dn are equal, so that the image input means V1 to Vn and the encoding devices D1 to Dn are equal to each other. Although illustrated as one-to-one, the number of the plurality of video input means V1 to Vn and the plurality of encoding devices D1 to Dn may be different from each other. Since each of the plurality of encoding apparatuses D1 to Dn generates the original video into the bitstreams BS1 to Bn of the H.264 / AVC format is disclosed as an image standard technology designated by JVT as described above, Detailed description will be omitted.

The plurality of image input means V1 to Vn may be omitted when the original image is pre-stored in a separate recording medium or a memory of the encoding apparatuses D1 to Dn.

The image synthesizing apparatus VC is a core component of the present invention, receives a plurality of bitstreams BS output from the plurality of encoding apparatuses D1 to Dn, and synthesizes the plurality of received bitstreams BS. The synthesized image is generated and the synthesized image is output as a synthesized bitstream (CVBS). In this case, the composite bitstream CVBS output from the image synthesizing apparatus VC is also configured according to the H.264 / AVC format.

The video synthesizing apparatus VC of the present invention is applied to the entire plurality of bit streams BS1 to Bn applied by the plurality of encoding apparatuses D1 to Dn similarly to Korean Patent No. 10-1053161 described in the prior art. Without performing decoding, only a part of each of the encoded bitstreams BS1 to Bn is extracted and modified, and the synthesized bitstream CVBS is generated by re-coding and inserting the modified region. Therefore, compared to the conventional technique of decoding each of the plurality of bitstreams D1 to Dn and restoring the image, synthesizing the plurality of restored images, and then encoding the synthesized bitstream (CVBS), the amount of work can be greatly reduced. In addition, low-cost, low-end hardware can quickly generate composite images.

However, the image synthesizing apparatus VC according to the present invention changes the MPM setting for some macroblocks in consideration of the arrangement position where the plurality of received bitstreams BS1 to BSn are arranged in the synthesized image, thereby generating a surrounding image after generating the synthesized image. MPM error occurring at the interface with the bitstreams BS1 to BSn disposed adjacent to the node can be eliminated.

In the above-described conventional technology, since only the parameters and variables are extracted from the plurality of bitstreams D1 to Dn, the synthesized bitstream CVS can be generated by simple combining. In this case, an error due to MPM mismatch occurs in the synthesized image as shown in FIG. 1.

In contrast, the image synthesizing apparatus VC of FIG. 3 determines an arrangement position where each of the plurality of bitstreams BS1 to BSn is to be arranged in the synthesized image, and based on the determined arrangement position, the image synthesis apparatus VC of FIG. It is determined whether there is a bitstream for another image disposed adjacent to the periphery. If there is a bitstream for another image, the intra prediction mode of the boundary region is reflected in the image of the currently placed bitstream in the image of the bitstream for the adjacently arranged image, thereby eliminating the image error due to the MPM mismatch. To be able.

4 shows the configuration of the image synthesizing apparatus of FIG.

In FIG. 4, the video synthesis apparatus VC includes a stream syntax analyzer 100, a parameter and header generator 200, a macroblock coded syntax corrector 300, and a stream synthesizer 400. Here, the stream parsing unit 100, the parameter and header generating unit 200, and the stream synthesizing unit 400 will briefly describe each operation in the configuration disclosed in the above-described image synthesis apparatus.

First, the stream parsing unit 100 receives a plurality of bit streams BS1 to BSn in units of NAL (Network Abstraction Layer), and analyzes NAL header syntax of the received bitstreams BS1 to BSn. Determine the data type. The data type may be a sequence parameter set (hereinafter referred to as SPS), a picture parameter set (hereinafter referred to as PPS), a slice, or the like. The parser performs parsing on the header of the slice and the parameters of the SPS and PPS according to the determined data type. At this time, the stream parsing unit 100 parses the encoded and transmitted bitstreams BS1 to BSn in the encoded state.

The stream parsing unit 100 may analyze the syntax of the SPS according to the authorized data type to determine the size of the input image, and may analyze the syntax of the PPS to determine the size and position of each slice. . The slice header may be analyzed to determine the position of the first macro block of the slice. That is, the stream syntax analyzer 100 may analyze syntaxes of a plurality of bitstreams to obtain macro block position information of slices constituting the image and the size of the image.

The parameter and header generator 200 generates the SPS and the PPS for the composite image by using the SPS and the PPS analyzed by the stream syntax analyzer 100 in the plurality of bitstreams BS1 to BSn. At this time, the SPS and PPS generated by the parameter and header generator 200 for the composite image are mostly used by merging the headers of the SPS and PPS analyzed in the input bitstreams BS1 to BSn. That is, the parameter and header generator 200 merges the parameters of the plurality of SPSs included in each of the input images and the plurality of PPSs to merge the plurality of input images into one composite image, and then converts the plurality of input images into SPSs and PPSs corresponding to the composite image. Create However, the parameter and header generation unit 200 changes or newly sets the parameters of the SPS, the PPS, and the parameters of some headers of the header of the slice to be suitable for the composite image. The parameter and header generation unit 200 may preset and store configuration information of the synthesized image in order to change or newly set a part of the parameters of the SPS and the PPS and the header of the slice. In some cases, configuration information of the composite image may be stored by receiving a user command from the outside. The configuration information of the synthesized image may include the overall size of the synthesized image, the size of each of the plurality of input images, and the arrangement position.

The parameter and the header of the parameter and slice of the SPS and PPS that the parameter generator 200 changes or generates may be referred to as a modified syntax, and a detailed description of generating the modified syntax is disclosed in the prior art, and thus, further description thereof will be omitted. .

The macroblock encoding syntax modifying unit 300 is a means for regenerating and re-encoding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or a transform coefficient encoded for the target macroblock ( Means for re-encoding the entropy encoding syntax including information on the number of non-zero transform coefficients in performing entropy encoding on a transform coefficient, by using any one or all of the means. The encoding syntax of the target macroblock can be recoded.

Here, the encoding syntax of the macroblock, that is, the macroblock encoding syntax, is used to mean a code string having a predetermined rule generated by encoding an image pixel signal value included in the macroblock. It may mean a code string encoded in a general sense.

5 is a detailed block diagram of the macroblock coded syntax corrector 300.

The macroblock coded syntax corrector 300 may include an entropy coded syntax recoding unit 300a and an MPM corrector 300b, and may further include a macroblock padding unit 300c as necessary. If necessary, the apparatus may further include a skip mode encoding syntax corrector 300d.

An entropy encoding syntax re-encoding unit 300a modifies the entropy encoding syntax including information on the number of non-zero transform coefficients in the entropy encoding for the target macroblock, thereby encoding the encoded syntax of the target macroblock. Reencode.

Here, the transform coefficient is a transform coefficient that is an object to perform entropy encoding. That is, entropy encoding is performed on a transform coefficient or a moving vector of a differential image obtained after predictive encoding is performed. The transform coefficient in the present invention refers to the transform coefficient. Herein, the conversion coefficient may be obtained by using various kinds of conversion methods, and may be obtained through, for example, DCT or Hadamard transformation.

Here, the entropy encoding syntax including information about the number of non-zero transform coefficients will be described in more detail below.

Herein, the target macroblock to which the entropy encoding syntax recoding unit 300a re-encodes the syntax is preferably a boundary macro block as described in detail below with reference to the PMV correction unit 300b. It is preferable to modify the entropy encoding syntax including information on the number of non-zero transform coefficients in entropy encoding of the boundary macroblock by referring to neighboring macroblocks located in the vicinity.

In this case, the entropy encoding syntax recoding unit 300a re-encodes the boundary macroblock among the target macroblocks, and performs the entropy encoding on the boundary macroblock using a variable length coding (VLC) method. Modifying the VLC table for the boundary macroblock using the neighboring macroblock of another input image disposed adjacent to the boundary macroblock, and using the modified VLC table, 0 of the transform coefficients of the boundary macroblock. It is preferable to re-encode the encoded syntax of the boundary macroblock by modifying the variable length encoded syntax including information on the number of transform coefficients.

Here, an error block of transform coefficients in H.264 / AVC may be encoded using CAVLC. The CAVLC may select the VLC look-up table for the level parameter according to the size of the recently encoded level and encode the transform coefficients of the current macroblock. The syntax encoded by the VLC includes TotalCoeffs, which is the number of non-zero coefficients, and coeff_token, which is a code string including information of TrailingOnes, which indicate the number of transform coefficients having an absolute value of 1, successively. .

For example, when encoding coeff_token for a 4x4 block, four look-up tables, three variable length code tables, and one fixed length code table may be used. Here, the table is selected by a non-zero number that exists in the previously coded left (Left) and upper block (Upper). If there is no existing macroblock, the table may be selected incorrectly.

Therefore, the entropy encoding syntax recoding unit 300a modifies the VLC table for the boundary macroblock using the peripheral macroblock of another input image disposed adjacent to the boundary macroblock, and uses the modified VLC table. By modifying the variable length coding syntax coeff_token including information on the number of non-zero transform coefficients of the transform macroblock of the boundary macroblock, it is preferable to re-encode the encoded syntax of the boundary macroblock.

The MPM corrector 300b includes, from each of the plurality of bitstreams, a plurality of boundary macroblocks, which are macroblocks arranged on the boundary of the input image, among the target macroblocks, and another adjacent to each of the plurality of macroblocks. Decodes and extracts a neighboring macroblock that is the macroblock of an input image, analyzes the extracted prediction macro set in each of the boundary macroblock and the neighboring macroblock, and according to a prediction mode of the neighboring macroblock, MPM (Most Probable Mode). ) Is re-encoded to re-encode the encoding syntax of the boundary macroblock.

The MPM corrector 300b is provided to prevent an error due to MPM mismatch in the synthesized image as shown in FIG. 1 by synthesizing the input image encoded by the existing synthesized image generation technology without decoding the synthesized image as it is. do. The MPM correction unit 300b analyzes the arrangement of each of the plurality of input images included in the synthesized image and determines the image boundary from the arrangement of the analyzed input images. The MPM correction unit 300b determines the boundary of each input image because, when the MPM mismatch error is composed of a plurality of input images, a macroblock that does not exist in the boundary of each input image is additionally disposed. Because it occurs.

The MPM corrector 300b extracts a macroblock disposed at the determined image boundary and a macroblock around the image boundary, and analyzes and re-encodes the MPMs of the extracted macroblocks and outputs the extracted macroblocks. That is, the MPM corrector 300b decodes some macroblocks disposed at the image boundary, corrects the MPM, and entropy the corrected MPM to solve the MPM mismatch error according to the arrangement of the plurality of input images to be included in the composite image. Encode and output In the present invention, for convenience of description, the flag bit of 1 bit and the additional bit of 3 bits for the MPM described above will be referred to as MPM. That is, the MPM to be corrected and encoded may be 1 bit of flag bits or 1 bit of flag bits and 3 bits of additional bits.

The macro block padding unit 300c pads an additional macro block to one side of the target macro block and encodes the padded macro block to generate an encoding syntax for the padded macro block. The padded macro block may be a macro block having a predetermined value as a signal value of a pixel.

When synthesizing the encoded images using the H.264 encoding method, a screen break occurs in the boundary portion between the images as described above. This is because, firstly, since the encoded bitstream data is used instead of directly encoding the synthesized image in the synthesis process, the final synthesized image does not properly reflect the encoding environment of each image before synthesis. This is because sub-pixel MV, which is an important factor in H.264 / AVC's standard intra mode prediction, is not taken into account. Finally, in inter mode prediction, the left (left) and the upper (upper) in the case of macroblocks at the boundary of an image This is because if there is no macroblock in the upper right, the value is filled with a constant size including 128. In order to prevent cracking occurring at the boundary surface caused by the above causes, the macro block padding unit 300c according to the present invention pads the macro block on one side of the boundary macro blocks of the image. The padded macro block may be used as a reference block of the boundary macro block.

In this case, the stream synthesis unit 400 further inserts a syntax for the padded macro block encoded by the macro block padding unit 300c, and outputs a bitstream of the synthesized image including the plurality of input images. can do.

A skip mode encoding syntax corrector 300d modifies a skip mode encoding syntax of the boundary macroblock, wherein macroblocks included in the other input image arranged adjacent to the image including the boundary macroblock are encoded in a skip mode. Based on the received information, the encoding syntax for the skip mode of the boundary macroblock is modified.

In H.264 / AVC inter-mode predictive encoding, there is a macro block that is skip mode coded, in which a motion vector is not separately calculated and transmitted.

For example, the SAD (sum of absolute difference) value is calculated using the difference between the previous macro block and the current macro block, and if there is little difference between the previous macro block and the current macro block based on the SAD information, the current macro block is calculated. It may be decided to encode in a skip mode.

Here, in determining whether to encode the macroblock in the skip mode, the skip mode encoding information of the block may be used as an adjacent neighboring key.

For example, in the inter mode prediction encoding, there is a skip mode encoding syntax called mb_skip, which is a value indicating how many macro blocks are protected in skip mode before the macro block to be currently encoded. The mb_skip syntax may be used to determine whether to encode a macroblock to be currently encoded in a skip mode.

However, when synthesizing a plurality of different images as in the present invention, mb_skip syntax information of a neighboring image may incorrectly affect the skip mode encoding determination of the corresponding image. That is, when a plurality of images are synthesized, when a macroblock of a neighboring image is encoded in a skip mode, a boundary macroblock of the corresponding image may be incorrectly encoded in a skip mode.

Accordingly, the skip mode encoding syntax corrector 300d encodes the boundary macroblock in the skip mode by using information encoded by the skip mode in the macroblock of the surrounding image other than the image including the boundary macroblock. It is desirable to modify the skip mode encoding syntax involved in the decision. Preferably, the number of encoded macroblocks is counted in a skip mode among macroblocks of a neighboring image, and the skipped mode encoding syntax of the boundary macroblock is modified using the counted number of macroblocks encoded in the skipped mode. . In this case, the modified skip mode encoding syntax may be mb_skip.

The stream synthesizing unit 400 generates and outputs a bit stream of the synthesized image by inserting the modified syntax generated by the parameter and header generation unit 200 into the plurality of input images by substituting the syntax of the received bit stream. The bitstream generated by the stream synthesizer 400 is also data encoded according to H.264 / AVC, and may generate a NAL unit (NALU) as a basic unit. Here, the stream synthesis unit 400 does not simply insert the modified syntax into the bitstream, but inserts the syntax re-encoded by the macroblock coded syntax corrector 300. For example, the MPM corrector 300b inserts an entropy-encoded MPM into a bitstream, thereby outputting a composite image from which an MPM mismatch error is eliminated.

FIG. 6 shows a detailed configuration of the MPM corrector 300b of FIG.

In FIG. 6, the MPM corrector 300b includes a boundary block extractor 310b, a prediction mode analyzer 320b, an MPM regenerator 330b, and a prediction mode recoder 340b.

The boundary block extractor 310b determines an image boundary according to an arrangement position where a plurality of input images are arranged in the synthesized image, and extracts a macroblock of the determined image boundary and a macroblock of adjacent images arranged around the image block.

The boundary block extractor 310b includes an image layout analyzer 311b, an image boundary determiner 313b, a boundary macroblock extractor 315b, and a neighboring macroblock extractor 317b. The image placement analyzer 311b analyzes a position where the input image of each of the plurality of bitstreams BS1 to BSn is to be arranged in the composite image through the configuration information of the composite image. The image boundary determination unit 313b determines the boundary between the generated images according to the position where the plurality of input images are arranged in the composite image. Here, the boundary between images does not simply mean an edge of one image, but a boundary between two images disposed adjacent to each other. That is, as shown in Fig. 1, in one image, since there is no image disposed adjacent to the periphery, there is no boundary between the images. However, in a synthesized image, boundaries between images occur as another image is disposed adjacent to one image. In this case, the boundary between the images in the specific image is variously changed according to the number, arrangement positions, etc. of the plurality of images included in the composite image. That is, in FIG. 1, as the image illustrated in (a) is disposed at the lower right side of the composite image including four images, the upper and left edges are determined as the boundary between the images. However, if the image is placed at the top left, the bottom and right edges are identified as the interface between the images.

As described above, however, in H.264 / AVC, the MPM is basically set by using a low number prediction mode among the prediction modes of the macroblocks disposed on the left and top of the macroblock. And the present invention is an invention for removing the error of the MPM. Therefore, the interface between the image of the current image and the adjacent image disposed on the right side may not be determined. That is, the boundary between the images may be determined only for the left end and the top of the current image. Here, the current image is an image corresponding to a slice output from the parameter and header generator 200 and means an input image corresponding to a bitstream to correct the current MPM.

The boundary macro block extractor 315b extracts a macro block disposed on an interface between images in the current image. The boundary macro block extractor 315b determines the size and position of each slice by parsing the PPS. If the determined position is a boundary between images, the boundary macro block extractor 315b decodes and extracts a macroblock included in the slice. . Hereinafter, in the present invention, the macro block disposed at the boundary between the images in the current image is called a boundary macro block. As described above, since the MPM calculation refers to the prediction mode of the macro block disposed on the left side and the upper side of the macro block, even if the macro blocks disposed on the right side and the lower side of the current image are disposed at the boundary between the images. It is not necessary to extract the boundary macroblocks.

The neighboring macroblock extractor 317b decodes and extracts a macroblock included in a boundary of an image disposed adjacent to the current image in the synthesized image. That is, the macroblock and the macroblock adjacent to the macroblock extracted by the boundary macroblock extraction unit 315b are decoded and extracted from the image disposed adjacent to the current image. In the present invention, the neighboring macroblock extracted by the neighboring macroblock extraction unit 317b is called a neighboring macroblock for convenience of description. That is, in the present invention, the neighboring macroblock refers to a macroblock of another image disposed adjacent to the boundary of the current image. And since the MPM is not calculated for the right and bottom boundaries of the current image, the macro blocks disposed on the top and left boundaries of other images do not need to be extracted even if they are arranged at the boundaries between the images.

The prediction mode analyzer 320b analyzes a prediction mode set in each of the macroblocks disposed at the boundary between the extracted current image and the adjacent images and the macroblocks of the adjacent images arranged at the periphery. The prediction mode analyzer 320b includes a current block prediction mode analyzer 321b and a neighboring block prediction mode analyzer 323b. The current block prediction mode analyzer 321b analyzes the prediction mode set in the boundary macroblock extracted by the boundary macroblock extractor 315b, and the neighboring block prediction mode analyzer 323b performs the neighboring macroblock extractor 317b. Analyze the prediction mode set in the neighboring macroblock extracted from. The analyzed prediction mode is transmitted to the MPM regenerator 330b.

The MPM regenerator 330b regenerates the MPM of the boundary macro block in consideration of the prediction mode of the analyzed surrounding macro block.

The MPM regenerator 330b selects a low number prediction mode among the prediction modes of the neighboring macroblocks analyzed by the neighboring block prediction mode analyzer 323b, and analyzes the selected prediction mode and the current block prediction mode analyzer 321b. It is determined whether the prediction mode of the bounded boundary macroblock is the same prediction mode.

If the selected prediction mode and the prediction macro of the boundary macroblock are the same, only one bit of the flag bit is set to regenerate the MPM. If the selected prediction mode and the prediction macro of the boundary macro block are different from each other, together with the flag bit of 1 bit, The MPM is regenerated by setting an additional three bits containing the prediction mode information. Here, the number of bits of the flag bit and the additional bit may vary depending on the settings of the image encoding apparatus and the decoding apparatus.

Thereafter, the prediction mode recoding unit 340b entropy-codes the regenerated MPM and transmits the entropy encoding to the stream synthesis unit 400.

That is, the image synthesizing system and the image synthesizing apparatus generate a composite image by modifying the SPS of the input image, the parameters of the PPS and the header of the slice in the plurality of input images applied to the bitstreams BS1 to BSn. Some macroblocks arranged at the interface are decoded, MPMs of the macroblocks are generated, encoded, and inserted into the synthesized image. Accordingly, a simple process may be added to the conventional technology of generating a composite image by modifying only the SPS of the input image, the parameters of the PPS, and the header of the slice, thereby providing a composite image from which an MPM error is eliminated.

Referring to FIGS. 3 to 6, the method for synthesizing the image of FIG. 7 firstly receives a plurality of bitstreams BS1 to BSn for each of the plurality of input images (S10). The stream parser 100 of the image synthesizing apparatus VC may receive a plurality of bitstreams BS1 to BSn in NAL units.

In addition, the stream parser 100 analyzes a syntax for each of the plurality of received bitstreams (S20). The stream parsing unit 100 first analyzes NAL header syntaxes of a plurality of bitstreams BS1 to BSn received in NAL units to determine a data type, and determines the parameters and slices of the SPS and PPS according to the determined data types. The header is parsed to determine the size of the input image, the size and position of each slice, and the position of the macro block.

Thereafter, the parameter and header generator 200 generates a parameter and a slice header for the synthesized image (S30). The parameter and header generation unit 200 merges and uses most of the headers of the parameters and slices of the SPS and PPS analyzed in the input bitstreams BS1 to BSn, but includes some headers of the headers of the parameters and slices of the SPS and PPS. The parameter of is changed or newly set to match the configuration information of the preset composite image.

And means for regenerating and re-encoding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or a transform coefficient encoded for the target macroblock. Means for re-encoding the entropy encoding syntax including information about the number of non-zero transform coefficients in performing entropy encoding on the target macroblock using any one or all of the means. The coded syntax is re-encoded (S40).

The macroblock encoding syntax correcting step (S40) may include a plurality of boundary macroblocks, which are macroblocks arranged on a boundary of the input image, from each of the plurality of bitstreams, and another input disposed adjacent to each of the plurality of macroblocks. Decode and extract the neighboring macroblocks that are the macroblocks of the image, analyze the extracted prediction macros set in each of the extracted boundary macroblocks and the neighboring macroblocks, and according to a prediction mode of the neighboring macroblocks, MPM (Most Probable Mode) MPM correction step (S41) of regenerating and re-encoding the data; And re-encoding the encoding syntax of the target macroblock by modifying the entropy encoding syntax including information on the number of non-zero transform coefficients in the entropy encoding for the target macroblock. Step S42; may include.

Here, the entropy encoding syntax recoding step (S42) re-encodes the boundary macroblock among the target macroblocks, and performs the entropy encoding on the boundary macroblock using a variable length coding (VLC) method. Modifying the VLC table for the boundary macroblock using the neighboring macroblock of another input image disposed adjacent to the boundary macroblock, and using the modified VLC table, 0 of the transform coefficients of the boundary macroblock. It is desirable to modify the variable length coding syntax, which indicates the number of transform coefficients, to re-encode the coding syntax of the boundary macroblock.

Next, the stream synthesis unit 400 inserts the modified syntax generated by the parameter and header generation unit 200 in the plurality of bit streams BS1 to BSn received by the image synthesis apparatus VC, and counts the number of macroblock encoding syntaxes. The macroblock encoding syntax re-encoded by the government 300 is inserted to output a composite video CVBS as a bitstream (S50).

The method according to the invention can be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and also include a carrier wave (for example, transmission through the Internet). The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible.

Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims

A stream parsing unit configured to receive bitstreams of a plurality of input images, and perform parsing by performing entropy decoding up to at least a macroblock layer on each of the plurality of received bitstreams;

A parameter and header generator configured to modify the syntax analyzed by the stream parser according to configuration information about a preset composite image to generate a modified syntax;

Means for regenerating and re-coding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or performing entropy encoding on a transform coefficient encoded for the target macroblock. Re-encoding the encoded syntax of the target macroblock using any one or all of the means for re-encoding the entropy encoded syntax including information on the number of non-zero transform coefficients. Macroblock encoding syntax correction; And

The synthesis including the plurality of input images by inserting the modified syntax generated by the parameter and header generation unit and the syntax re-encoded by the macroblock encoding syntax correction unit by replacing the existing syntax included in the bitstream. A stream synthesizer for outputting a bitstream of an image; Including,

The macroblock encoding syntax correction unit,

From each of the plurality of bitstreams, a plurality of boundary macroblocks which are macroblocks arranged at the boundary of the input image among the target macroblocks, and the macroblocks of other input images arranged adjacent to each of the plurality of macroblocks. Decode and extract a neighboring macroblock, analyze a prediction mode set in each of the extracted boundary macroblock and the neighboring macroblock, regenerate a Most Probable Mode (MPM) according to the prediction mode of the neighboring macroblock, An MPM correction unit for re-encoding the encoding syntax of the macro block; And

An entropy encoding syntax re-encoding unit for re-encoding the encoding syntax of the target macroblock by modifying the entropy encoding syntax including information about the number of non-zero transform coefficients in the entropy encoding for the target macroblock; Image synthesizing apparatus comprising a.
According to claim 1,

The macroblock encoding syntax corrector further includes a macroblock padding unit configured to pad an additional macroblock to one side of the target macroblock and to encode the padded macroblock to generate an encoding syntax for the padded macroblock.

The stream synthesizing unit further inserts an encoding syntax for the padded macro block encoded by the macro block padding unit, and outputs a bitstream of the synthesized image including the plurality of input images. Device.
According to claim 1,

The entropy encoding syntax re-encoding unit re-encodes the boundary macroblock among the target macroblocks and performs the entropy encoding using the variable length coding (VLC) method for the boundary macroblock. Modifying the VLC table for the boundary macro block using the neighboring macro block of another input image arranged adjacent to the macro block, and using the modified VLC table, the non-zero of the transform coefficients of the boundary macro block. And encoding the encoded syntax of the boundary macroblock by modifying the variable length encoded syntax including information on the number of transform coefficients.
The method of claim 1,

The entropy encoding syntax re-encoding unit modifies a skip mode encoding syntax of the boundary macroblock, wherein the macroblocks included in the other input image arranged adjacent to the image including the boundary macroblock are encoded in skip mode. And a skip mode encoding syntax corrector for correcting the encoding syntax for the skip mode of the boundary macroblock.
According to claim 1,

The MPM corrector

A boundary block extracting unit for determining a boundary surface of an image according to an arrangement position where the plurality of input images are arranged in the composite image, and decoding and extracting the boundary macro block and the neighboring macro block from the determined boundary surface;

A prediction mode analyzer configured to analyze and determine the prediction mode set in each of the boundary macro block and the neighboring macro block;

MPM reproduction which determines the MPM prediction mode of the boundary macroblock by using the prediction mode set in the neighboring macroblock, regenerates the MPM by determining whether the prediction mode of the boundary macroblock and the recalculated MPM prediction mode are the same. part; And

A prediction mode recoding unit encoding the reproduced MPM to be inserted into a bitstream of the synthesized image; Image synthesizing apparatus comprising a.
The method of claim 1, wherein the boundary block extraction unit

An image arrangement analyzer configured to determine an arrangement position of each of the plurality of input images in the synthesized image according to the configuration information of the synthesized image set in the parameter and the header generator;

An image boundary discrimination unit which determines an interface between adjacent images generated according to a position where a plurality of input images are arranged in the composite image;

A boundary macroblock extracting unit configured to decode and extract the boundary macroblocks disposed on an interface between the images in a current image for correcting an MPM among the plurality of input images; And

A neighboring macroblock extraction unit configured to decode and extract the neighboring macroblock disposed adjacent to the boundary macroblock from images disposed adjacent to the current image; Image synthesizing apparatus comprising a.
The method of claim 6, wherein the prediction mode analysis unit

A current block prediction mode analyzer configured to analyze and determine the prediction mode set in the boundary macro block; And

A neighboring block prediction mode analyzer configured to analyze and determine the prediction mode set in the neighboring macroblock; Image synthesizing apparatus comprising a.
The method of claim 6, wherein the MPM regeneration unit

The MPM prediction mode is determined by determining a prediction mode having the lowest assigned number among the prediction modes of each of the neighboring macroblocks.If only the prediction mode of the MPM prediction mode and the boundary macroblock is the same, only flag bits are set. And setting an additional bit for designating a prediction mode of the boundary macroblock together with the flag bit when the MPM prediction mode and the prediction mode of the boundary macroblock are different from each other.
The method of claim 8, wherein the MPM regeneration unit

And the prediction mode having the lowest number among the prediction modes of the neighboring macroblocks disposed adjacent to the left and the top of the boundary macroblock as the MPM prediction mode.
The method of claim 1, wherein the parameter and header generating unit

And a modified syntax is generated by modifying an SPS, a parameter of a PPS, and a header of a slice among the syntaxes of the plurality of bit streams.
The method of claim 1, wherein the parameter and header generating unit

And synthesizing the configuration information of the synthesized image from the outside.
A plurality of encoding devices each receiving a different original image, encoding the predetermined original encoding method, and outputting a bitstream; And

Receives a plurality of input images as the plurality of bitstreams, modifies the syntax for each of the plurality of bitstreams according to configuration information of a predetermined composite image, generates a modified syntax, and generates an existing syntax included in the bitstream. And inserting a subfield, generating a bitstream for the synthesized image, wherein the plurality of boundary macroblocks are macroblocks arranged at the boundary of the input image from each of the plurality of bitstreams, and adjacent to each of the plurality of macroblocks. Decoding and extracting the neighboring macroblocks that are the macroblocks of the other input image arranged, and analyzing the extracted boundary macroblock and the prediction mode set in each of the neighboring macroblocks, and analyzing the MPM according to the prediction mode of the neighboring macroblock. Regenerate and recode Most Probable Mode Video synthesizer that is inserted into the bitstream for the image; Including,

The image synthesizing apparatus

A stream parsing unit configured to receive the plurality of bitstreams and perform parsing;

A parameter and header generator for modifying the syntax analyzed by the stream parser to generate the modified syntax;

Means for regenerating and re-coding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or performing entropy encoding on a transform coefficient encoded for the target macroblock. Re-encoding the encoded syntax of the target macroblock using any one or all of the means for re-encoding the entropy encoded syntax including information on the number of non-zero transform coefficients. Macroblock encoding syntax correction; And

Replacing the existing syntax included in the bitstream by inserting the modified syntax generated by the parameter and header generation unit and the syntax re-encoded by the macroblock coded syntax corrector into the bitstream for the composite image, A stream synthesizer configured to output a bitstream of the synthesized image including two input images; Image synthesis system comprising a.
The method of claim 12,

The macroblock encoding syntax correction unit,

From each of the plurality of bitstreams, a plurality of boundary macroblocks which are macroblocks arranged at the boundary of the input image among the target macroblocks, and the macroblocks of other input images arranged adjacent to each of the plurality of macroblocks. Decode and extract a neighboring macroblock, analyze a prediction mode set in each of the extracted boundary macroblock and the neighboring macroblock, regenerate a Most Probable Mode (MPM) according to the prediction mode of the neighboring macroblock, An MPM correction unit for re-encoding the encoding syntax of the macro block;

An entropy encoding syntax re-encoding unit for re-encoding the encoding syntax of the target macroblock by modifying the entropy encoding syntax including information about the number of non-zero transform coefficients in the entropy encoding for the target macroblock; And

A skip mode encoding syntax of the boundary macroblock is modified, but macroblocks included in the other input image disposed adjacent to the image including the boundary macroblock are encoded using a skip mode, and the boundary macroblock is used. And a skip mode coded syntax corrector for correcting the coded syntax for the skip mode of the skip mode.
The method of claim 12, wherein the image synthesis system

A plurality of image input means for generating the original image and transmitting the original image to a corresponding one of the plurality of encoding devices; Image synthesizing system further comprises.
The method of claim 12, wherein the image synthesis system

At least one recording medium in which the plurality of original images are pre-stored and transmitted to a corresponding encoding device among the plurality of encoding devices; Image synthesizing system further comprises.
An image synthesizing method of an image synthesizing apparatus of an image synthesizing apparatus comprising a stream parser, a parameter and header generator, a macroblock encoding syntax corrector, and a stream synthesizer,

Receiving, by the stream parser, a plurality of bitstreams for each of a plurality of input images, and analyzing the syntax for each of the received bitstreams;

Generating a modified syntax by modifying the syntax analyzed by the parameter and the header generator according to configuration information about a preset composite image;

Means for regenerating and re-encoding the Most Probable Mode (MPM) of each target macroblock included in each of the plurality of bitstreams, or a transform coefficient encoded for the target macroblock. Means for re-encoding the entropy encoding syntax including information on the number of non-zero transform coefficients in performing entropy encoding on the target macroblock by using any one or all of the means. Macroblock encoding syntax modification step of re-encoding encoding syntax; And

The stream synthesis unit includes the plurality of input images by inserting the modified syntax generated by the parameter and header generation unit and the syntax re-encoded by the macroblock encoding syntax correction unit by replacing the existing syntax included in the bitstream. Outputting a bitstream for the synthesized video; Image synthesis method comprising a.
The method of claim 16, wherein the modifying macroblock encoding syntax comprises:

Decode a plurality of boundary macroblocks, which are macroblocks arranged at the boundary of the input image, from each of the plurality of bitstreams, and peripheral macroblocks that are the macroblocks of another input image disposed adjacent to each of the plurality of macroblocks Extracting, analyzing a prediction mode set in each of the extracted boundary macroblock and the neighboring macroblock, and regenerating and re-encoding a Most Probable Mode (MPM) according to the prediction mode of the neighboring macroblock; And

Re-encoding the encoding syntax of the target macroblock by modifying the entropy encoding syntax including information on the number of non-zero transform coefficients in the entropy encoding for the target macroblock. Image synthesis method comprising a.
The method of claim 17,

In the entropy encoding syntax recoding step, the boundary macroblock of the target macroblock is recoded, and the boundary macroblock is entropy encoded by using a variable length coding (VLC) method. The VLC table for the boundary macroblock is modified using the neighboring macroblock of another input image disposed adjacent to the macroblock, and the nonzero conversion of the transform coefficients of the boundary macroblock is performed using the modified VLC table. And re-encoding the encoded syntax of the boundary macroblock by modifying the variable length encoded syntax representing the number of coefficients.
18. The method of claim 17, wherein the MPM correction step is

Decode and extract the macro block, determine the arrangement position of each of the plurality of input images in the composite image according to the configuration information of the composite image, and generate according to the position where the plurality of input images are arranged in the composite image Determine an interface between adjacent images, decode and extract the boundary macroblocks disposed on the interface between the images from a current image for correcting an MPM among the plurality of input images, and adjacent to the current image And decoding and extracting the neighboring macroblock disposed adjacent to the boundary macroblock from the arranged images.
18. The method of claim 17, wherein the MPM correction step is

Analyze and determine the prediction mode set in the boundary macroblock, analyze and determine the prediction mode set in the neighboring macroblock, and determine the prediction mode with the lowest assigned number among the prediction modes of each of the neighboring macroblocks. Determine the MPM prediction mode, determine whether the prediction mode of the boundary macroblock and the determined MPM prediction mode are the same, regenerate the MPM, and insert the regenerated MPM into the bitstream for the synthesized image. And encoding and re-encoding the MPM.
The method of claim 20, wherein the MPM correction step

The MPM prediction mode is determined by selecting the prediction mode having the lowest assigned number among the prediction modes of the neighboring macroblocks disposed adjacent to the left and the top of the boundary macroblock as the MPM prediction mode. Synthetic method.
The method of claim 20, wherein the MPM correction step

If the MPM prediction mode and the prediction mode of the boundary macroblock are the same, only flag bits are set, and if the MPM prediction mode and the prediction mode of the boundary macroblock are different from each other, prediction of the boundary macroblock with the flag bit is performed. And regenerating the MPM by setting an additional bit for designating a mode.