KR101446074B1 - Method, apparatus and system for combining plural picutres - Google Patents

Abstract

A method, an apparatus and a system for synthesizing an image are disclosed. The apparatus for synthesizing an image of the present invention includes: a stream syntax analysis unit to receive a bit stream of an input image and analyze a syntax of each of the received bit streams; a parameter and header generation unit to modify the syntax analyzed by the stream syntax analysis unit according to configuration information of a predetermined synthetic image to generate a modified syntax; a most probable mode (MPM) correction unit to decode and extract a plurality of boundary macro blocks, which are arranged at a boundary of the input image, and a periphery macro block of a different input image, which is arranged next to each of the macro blocks, from each of the bit streams, and analyze a predictive mode set to each of the extracted boundary macro blocks and the periphery macro block to regenerate and re-encode an MPM according to the predictive mode of the periphery macro block; and a stream synthesis unit to replace an existing syntax included in the bit stream by inserting the modified syntax generated by the parameter and header generation unit and the regenerated and encoded MPM in order to output the bit stream of the synthesized image including the input images.

Description

TECHNICAL FIELD [0001] The present invention relates to an image synthesizing method,

The present invention relates to a video synthesis method, apparatus, and system, and more particularly, to a video synthesis method, apparatus, and system capable of solving a maximum probability mode mismatch in an inter-picture coding mode at the time of video synthesis.

International standardization organizations for video signal compression include the International Telecommunication Union Telecommunication Standardization Sector (ITU-T), which is established for video applications such as video conferencing, and ISO / IEC for various applications such as video data storage or broadcasting . In addition, ITU-T and ISO / IEC together form JVT (Joint Video Team), which provides compression performance higher than existing Moving Picture Experts Group (MPEG) -2, H.263, and MPEG-4 Visual Video compression coding standards. 264 / AVC video coding standard. H.264 / AVC is currently regarded as a next-generation video compression technology. Especially, it is widely applied to multimedia multimedia services such as multi-channel high-quality image compression, Internet, cable modem, video transmission in mobile communication network, digital data broadcasting, etc., while being combined with next generation multimedia services such as digital TV, satellite and terrestrial DMB (Digital Multimedia Broadcasting) .

The H.264 / AVC based high-speed image synthesis technique is a technique of generating a combined image by combining plural independent images. One of the conventional high-speed image synthesis techniques is disclosed in Korean Patent No. 10-1053161 (published Jan. 28, 2011). A feature of the disclosed prior art is that a plurality of images are synthesized into a single image and then a bitstream for transmission is generated through an encoding process, but a plurality of bitstreams for each of the already generated images are received Extracts and corrects only a specific portion of each of the plurality of received bitstreams, and generates a bitstream for one combined image.

That is, the conventional art disclosed in the video synthesis technique in the H.264 / AVC compression region does not perform decoding on a plurality of inputted input video bitstreams. Instead, a sequence parameter set (SPS), a picture parameter set (PPS), and a slice header included in each of a plurality of input image bitstreams are parsed. Then, the sequence parameter set (SPS), the picture parameter set (PPS), and the slice header parameters and variables of the plurality of input image bitstreams are changed or newly set so as to correspond to the display form of the composite image. That is, according to the prior art, a plurality of input image bitstreams are parsed, parameters are changed, and parameters are changed in a coded compressed region without decoding a plurality of input image bitstreams that are encoded and compressed, And generates and outputs a bitstream of the H.264 / AVC format.

Accordingly, it is possible to greatly reduce the decoding process for a plurality of input image bitstreams and the encoding process for a synthesized image bitstream, thereby generating a composite image at low complexity and high speed.

1 is a view showing a problem of a conventional image synthesis technique.

In FIG. 1, (a) represents one of a plurality of input images before generation of the composite image, and (b) represents a composite image including the input image of (a). As shown in FIG. 1 (a), each of the plurality of input images before being synthesized does not have other images around. Therefore, there is no other macroblock in the boundary of the boundary of the input image. However, in the composite image shown in (b) of FIG. 1, the input image of (a) is arranged at the lower right of the composite image. (a) is arranged at the lower right of the synthesized image, other input images that were not present before synthesis are disposed adjacent to the left boundary surface and the upper boundary boundary of the input image of (a) do.

FIG. 2 shows nine prediction modes for intraprediction encoding.

In H.264 / AVC, nine prediction modes are provided as shown in FIG. 2 in intraprediction coding for improving the compression ratio, and one of the modes is selected to generate and encode a prediction block. In FIG. 2, the number assigned to each prediction mode is a number assigned according to the occurrence probability of each prediction mode, and a lower number means higher probability of occurrence. The image encoding apparatus encodes the prediction mode information together with the prediction block and transmits the prediction mode information, and requires four bits to transmit the nine prediction modes. However, H.264 / AVC provides additional Most Probable Mode (MPM) to reduce the number of bits for transmitting prediction mode information. The MPM determines the highest probability mode by referring to the prediction mode of the neighboring blocks in intra prediction coding. If the prediction mode and the highest probability mode are the same, only one flag bit is set to express the flag bit to reduce the number of data bits to be transmitted. If the prediction mode is not the same as the optimum mode, the flag bit indicates that the prediction mode and the highest probability mode are not the same, and the additional 3 bits are added to obtain one of the remaining eight prediction modes excluding the prediction mode selected in the highest probability mode Lt; / RTI > That is, if the prediction mode and the highest probability mode are the same, only one bit is used to reduce the number of data bits to be transmitted. Even if the prediction mode and the highest probability mode are not the same, only 4 bits are required. Respectively.

The MPM defined in H.264 / AVC currently provides an A prediction mode that refers to a block disposed adjacent to the left side and a B prediction mode that refers to a block disposed adjacent to the top, and the A prediction mode and the B prediction mode And the prediction mode of the lower number among the mode numbers is selected by the MPM. Considering the MPM, the conventional image synthesis technique of FIG. 1 will be described. In (a), since there is only one input image, the prediction mode for the upper left macroblock is the average value mode ). Therefore, MPM is also set to DC.

In (b), since the other images are arranged adjacent to the left and top of the input image, the A prediction mode of the MPM is a vertical mode (VER) having a prediction mode 0 according to the prediction mode of the left macro block, Is a horizontal mode (HOR) in prediction mode 1 according to the prediction mode of the upper macroblock. Since the A prediction mode is the prediction mode number 0 and the B prediction mode is the prediction mode number 1, the A prediction mode with the lower mode number is selected as the MPM, and the MPM is set to the prediction mode 0 with the vertical mode (VER). Since this is different from the DC mode which is the MPM in the single image, the MPM mismatch problem occurs for the corresponding macroblock in the composite image.

The inconsistency of the MPM generates an unintentional motion vector value at the time of decoding the macroblock, resulting in deterioration of the picture quality or, in some cases, the syntax information exceeds the set limit, thereby stopping the operation of the decoder .

This problem of inconsistency of the MPM occurs because the prior art does not sufficiently consider the macroblock data at the time of generating the composite image, and it must be corrected in order to generate an error-free composite image.

It is an object of the present invention to provide an image synthesizing apparatus which generates a synthesized image at low complexity and high speed without causing an error due to an MPM mismatch.

Another object of the present invention is to provide a video synthesis system for achieving the above object.

It is still another object of the present invention to provide an image synthesis method for achieving the above object.

According to an aspect of the present invention, there is provided a video synthesis apparatus comprising: a stream parsing unit for receiving a bitstream of a plurality of input images and performing a syntax analysis for each of the plurality of received bitstreams; A parameter and a header generating unit for modifying the syntax analyzed by the stream parsing unit according to configuration information for a predetermined composite image to generate a modification syntax; A plurality of boundary macroblocks which are macroblocks arranged at a boundary of the input image from each of the plurality of bitstreams and a neighboring macroblock which is the macroblock of another input image arranged adjacent to each of the plurality of macroblocks are decoded An MPM correcting unit for MPM (Most Probable Mode) according to a prediction mode of the neighboring macroblock by analyzing a prediction mode set in each of the boundary macroblock and the neighboring macroblock; And inserting the modified syntax generated in the parameter and header generating unit and the regenerated and encoded MPM into a bitstream of the composite image including the plurality of input images, A stream synthesizer for outputting a stream; .

The MPM correcting unit may include a boundary block extracting unit for discriminating the boundary of the image according to the arrangement position in which the plurality of input images are arranged in the synthesized image and decoding and extracting the boundary macroblock and the neighboring macroblock from the discriminated boundary, ; A prediction mode analyzer for analyzing and discriminating the prediction mode set in each of the boundary macroblock and the neighboring macroblock; Determining whether the MPM prediction mode of the boundary macroblock is identical to the recalculated MPM prediction mode using the prediction mode set in the neighboring macroblock, part; And a prediction mode re-encoding unit that encodes the regenerated MPM so that the regenerated MPM can be embedded in the bitstream of the synthesized image. And a control unit.

Wherein the boundary block extracting unit comprises: an image arrangement analyzing unit for determining the arrangement position of each of the plurality of input images in the composite image according to the parameter and the configuration information for the composite image set in the header generating unit; An image boundary determining unit for determining an interface between adjacent images generated according to a position where a plurality of input images are arranged in the composite image; A boundary macroblock extracting unit which decodes and extracts the boundary macroblocks arranged at the boundary between the images in the current image for correcting the MPM among the plurality of input images; And a neighboring macroblock extracting unit for decoding and extracting the neighboring macroblocks arranged adjacent to the boundary macroblocks in images arranged adjacent to the current image. And a control unit.

The MPM regenerating unit discriminates the MPM prediction mode by discriminating the prediction mode having the lowest assigned number among the prediction modes of the neighboring macroblocks. If the prediction modes of the MPM prediction mode and the boundary macroblock are the same, And setting an additional bit for designating a prediction mode of the boundary macroblock together with the flag bit when the MPM prediction mode and the prediction mode of the boundary macroblock are different from each other.

According to another aspect of the present invention, there is provided an image synthesizing system including: a plurality of encoding devices for receiving different original images, encoding them using a predetermined encoding scheme, and outputting bitstreams; And a modifying unit configured to receive a plurality of input images as the plurality of bitstreams, modify the syntax of each of the plurality of bitstreams according to configuration information for a predetermined synthesized image, A plurality of boundary macroblocks, which are macroblocks arranged at boundaries of the input image from each of the plurality of bitstreams, and a plurality of boundary macroblocks, Decoding the neighboring macroblocks, which are the macroblocks of other input images arranged adjacent to each other, and extracting the extracted neighboring macroblocks; analyzing a prediction mode set in each of the extracted boundary macroblocks and the neighboring macroblocks; MPM (Most Probable Mode) is regenerated, re-encoded, An image synthesizer for inserting the synthesized image into a bitstream; .

According to another aspect of the present invention, there is provided an image synthesis method of a video synthesis apparatus of a video synthesis apparatus including a stream syntax analysis unit, a parameter and header generation unit, an MPM correction unit, and a stream synthesis unit, Receiving the plurality of bit streams for each of the plurality of input images, and analyzing a syntax for each of the received bit streams; Generating a modified syntax by modifying the parsed and parsed header according to configuration information for a predetermined composite image; Wherein the MPM correcting unit comprises: a plurality of boundary macroblocks, which are macroblocks arranged at the boundaries of the input image from each of the plurality of bitstreams; and a boundary macroblock which is a macroblock of another input image arranged adjacent to each of the plurality of macroblocks, Decoding and extracting the block; Analyzing a prediction mode set in each of the boundary macroblock and the neighboring macroblock from which the MPM correction unit has extracted and regenerating and re-encoding an MPM (Most Probable Mode) according to a prediction mode of the neighboring macroblock; And the stream synthesis unit replaces the existing syntax included in the bitstream and inserts the modification syntax and the encoded MPM generated in the parameter and header generation unit into the bitstream of the composite image including the plurality of input images, ; And a

Therefore, the image synthesis method, apparatus, and system of the present invention are capable of modifying parameters and variables in a coded compressed state without decoding and encoding a plurality of input image bitstreams, It is possible to eliminate errors and image quality deterioration of a composite image due to a prediction mode mismatch in a maximum probability mode.

1 is a view showing a problem of a conventional image synthesis technique.
FIG. 2 shows nine prediction modes for intraprediction encoding.
3 is a block diagram schematically illustrating an image synthesis system according to an embodiment of the present invention.
Fig. 4 shows a configuration of the image synthesizing apparatus of Fig.
FIG. 5 shows the detailed configuration of the MPM correction unit of FIG.
6 illustrates an image synthesis method according to an embodiment of the present invention.

In order to fully understand the present invention, operational advantages of the present invention, and objects achieved by the practice of the present invention, reference should be made to the accompanying drawings and the accompanying drawings which illustrate preferred embodiments of the present invention.

Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings. However, the present invention can be implemented in various different forms, and is not limited to the embodiments described. In order to clearly describe the present invention, parts that are not related to the description are omitted, and the same reference numerals in the drawings denote the same members.

Throughout the specification, when an element is referred to as "including" an element, it does not exclude other elements unless specifically stated to the contrary. The terms "part", "unit", "module", "block", and the like described in the specification mean units for processing at least one function or operation, And a combination of software.

3 is a block diagram schematically illustrating an image synthesis system according to an embodiment of the present invention.

Referring to FIG. 3, the image synthesis system of the present invention includes a plurality of image input means V1 to Vn, a plurality of encoding devices D1 to Dn, and a video synthesizer VC.

The plurality of image input means V1 to Vn are implemented by a camera or the like to generate an original image and transmit the original image to the corresponding encoding devices D1 to Dn of the plurality of encoding devices D1 to Dn.

Each of the plurality of encoding devices D1 to Dn receives an original image from a corresponding one of the plurality of image input means V1 to Vn and encodes the original image in accordance with the H.264 / AVC format, To Bn) to the image synthesizer (VC). 3, the number of image input means (V1 to Vn) and the number of image encoding means (D1 to Dn) are the same for convenience of description, The number of image input means V1 to Vn and the number of the plurality of encoding devices D1 to Dn may be different from each other. Since each of the plurality of encoding apparatuses D1 to Dn generates the original image in the H.264 / AVC format as bit streams BS1 to Bn, as described above, is a technique disclosed as an image standard technology specified by JVT, A detailed description thereof will be omitted.

The plurality of image input means V1 to Vn may be omitted when the original image is stored in a memory of a separate recording medium or the encoding devices D1 to Dn.

The video synthesizer VC is a core component of the present invention. The video synthesizer VC receives a plurality of bit streams BS output from a plurality of coders D1 to Dn and synthesizes the received plurality of bit streams BS , Generates a composite image, and outputs the composite image as a composite bitstream (CVBS). At this time, the composite bitstream (CVBS) output from the video synthesizer VC is also configured according to the H.264 / AVC format.

The image synthesizing apparatus VC of the present invention can be applied to all of a plurality of bit streams BS1 to Bn applied to a plurality of encoding apparatuses D1 to Dn in a similar manner to Korean Patent No. 10-1053161, A composite bitstream CVBS is generated by extracting and modifying only a part of each of the encoded bitstreams BS1 to Bn and re-encoding and inserting the modified area. Therefore, compared to the conventional technique of decoding all of the bit streams D1 to Dn to reconstruct an image, combining a plurality of reconstructed images, and then encoding the combined reconstructed bitstreams into a composite bitstream (CVBS), the workload can be greatly reduced , It is possible to quickly generate composite images even with low-cost and low-cost hardware.

However, the image synthesizer VC according to the present invention changes the MPM setting for some macroblocks in consideration of the arrangement position where the received bitstreams BS1 to BSn are arranged in the composite image, (BS1 to BSn) adjacent to the bit stream (BS1 to BSn).

In the above conventional technique, only a parameter and a variable are extracted from a plurality of bit streams D1 to Dn and a composite bitstream (CVBS) can be generated by simple combining. Therefore, And thus an error due to the MPM mismatch occurred in the composite image as shown in FIG.

In contrast, the image synthesizing apparatus VC of FIG. 3 discriminates the arrangement position where each of the plurality of bit streams BS1 to BSn is to be arranged in the composite image, and according to the determined placement position, And determines whether or not there is a bitstream for another image arranged adjacent to the periphery. If a bitstream for another image is present, the intra-prediction mode of the boundary region is reflected on the bitstream image of the current bitstream in the image of the bitstream for the adjacent image, thereby eliminating the image error due to the MPM mismatch .

Fig. 4 shows a configuration of the image synthesizing apparatus of Fig.

4, the image synthesizing apparatus VC includes a stream parsing unit 100, a parameter and header generating unit 200, an MPM correcting unit 300, and a stream synthesizing unit 400. [ Here, the operations of the stream parsing unit 100, the parameter and header generating unit 200, and the stream synthesizing unit 400 will be briefly described with the configuration disclosed in the above-described conventional image synthesizing apparatus.

First, the stream parsing unit 100 receives a plurality of bit streams BS1 to BSn on a network abstraction layer (NAL) basis, analyzes the NAL header syntax of the received bit streams BS1 to BSn Determine the data type. Here, the data type may be a sequence parameter set (SPS), a picture parameter set (PPS), a slice, or the like. Then, the SPS, PPS parameters and slice header are parsed according to the determined data type. At this time, the stream parsing unit 100 parses the encoded bit streams BS1 to BSn as they are in the encoded state.

The stream parsing unit 100 can analyze the syntax of the SPS according to the applied data type to determine the size of the input image and analyze the syntax of the PPS to determine the size and position where each slice is to be placed . Then, the position of the first macroblock of the slice can be determined by analyzing the slice header. That is, the stream parsing unit 100 may analyze the syntax of the plurality of bitstreams to obtain the size of the image and the macroblock location information of the slices constituting the image.

The parameter and header generation unit 200 generates the SPS and PPS for the composite image using the SPS and PPS analyzed by the stream parsing unit 100 in the plurality of bit streams BS1 to BSn. At this time, SPS and PPS generated by the parameter and header generation unit 200 for the combined image are mostly used by merely integrating the SPS, PPS parameters and slice header analyzed in the input bitstreams BS1 to BSn. That is, in order to merge the plurality of input images into one composite image, the parameter and header generation unit 200 merges the parameters of the plurality of SPSs and the plurality of PPSs included in each of the input images into the SPS and the PPS corresponding to the composite image . However, the parameter and header generation unit 200 changes or newly sets the parameters of the SPS, the PPS, and some of the headers of the header of the slice to fit the composite image. The parameter and header generation unit 200 may store the configuration information of the composite image in advance so as to change the parameters of the SPS and the PPS and a part of the header of the slice or set a new one. In some cases, the configuration information of the composite image can be stored by receiving a user command from the outside. Herein, the composition information of the composite image may include the total size of the composite image and the size and arrangement position of each of the plurality of input images.

The SPS and PPS parameters and the header of the slice that are changed or generated by the parameter and header generation unit 200 can be referred to as a modification syntax. A detailed description for generating a modification syntax is disclosed in the prior art, .

The MPM correcting unit 300 synthesizes the encoded input image in the encoded state without decoding the encoded input image so as to prevent an error caused by the MPM mismatch in the synthesized image as shown in FIG. do. The MPM correction unit 300 analyzes the arrangement of each of the plurality of input images included in the composite image, and determines the image boundary from the arrangement of the analyzed input images. The MPM correcting unit 300 determines the boundary of each input image when a plurality of input images are synthesized into a composite image due to an MPM mismatch error, macroblocks not existing in the boundary of each input image are additionally disposed .

Then, the MPM correction unit 300 extracts the macroblocks arranged at the identified image boundary and the macroblocks around the image boundary, analyzes and re-encodes the MPMs of the extracted macroblocks, and outputs the MPMs. That is, the MPM correcting unit 300 decodes some macroblocks arranged at the image boundary to correct an MPM mismatch error according to the arrangement of a plurality of input images to be included in the synthesized image, corrects the MPM, And outputs it. In the present invention, for convenience of description, the 1-bit flag bit and the 3-bit additional bit for the MPM described above will be referred to as MPM. That is, the MPM to be corrected and encoded may be a flag bit of 1 bit, a flag bit of 1 bit, and an additional bit of 3 bits.

The stream combining unit 400 generates a bit stream of the composite image by substituting the parameter and the modification sentence modified or generated by the header generating unit 200 into the syntax of the received bit stream to generate a bit stream of the composite image. The bit stream generated by the stream synthesizing unit 400 may also be generated based on the NAL unit (NALU) as data encoded according to H.264 / AVC. Here, the stream synthesizer 400 outputs the synthesized image from which the MPM mismatch error has been eliminated by inserting the entropy-encoded MPM into the bitstream, rather than merely inserting the modified syntax into the bitstream.

FIG. 5 shows the detailed configuration of the MPM correction unit of FIG.

5, the MPM correction unit 300 includes a boundary block extraction unit 310, a prediction mode analysis unit 320, an MPM reproducibility unit 330, and a prediction mode recoding unit 340.

The boundary block extracting unit 310 determines an image boundary according to a placement position where a plurality of input images are arranged in the composite image, and extracts a macro block of the determined image boundary and a neighboring image macro block.

The boundary block extracting unit 310 includes an image arrangement analyzing unit 311, an image boundary determining unit 313, a boundary macroblock extracting unit 315, and a neighboring macroblock extracting unit 317. The image arrangement analyzing unit 311 analyzes the positions where the input images of the plurality of bit streams BS1 to BSn are arranged in the composite image through the composition information of the composite image. The image boundary determining unit 313 determines the boundary between the images generated according to the positions of the plurality of input images in the composite image. Here, the boundary between images does not simply mean the edge of one image but the boundary between two images arranged adjacent to each other. That is, as shown in FIG. 1, since there is no image arranged adjacent to the periphery in one image, there is no boundary between the images. However, in a composite image, a boundary between images occurs due to the placement of another image adjacent to one image. At this time, the boundaries between the images in the specific image are variously changed according to the number of the plurality of images included in the composite image, the arrangement position, and the like. That is, in FIG. 1, the image shown in (a) is arranged at the lower right of the composite image including four images, so that the upper and lower edges are discriminated as the interface between the images. However, if the image is placed in the upper left corner, the bottom edge and the right edge are determined as the interface between the images.

However, as described above, H.264 / AVC is basically defined to set the MPM using the prediction mode of the lower number among the prediction modes of the macroblocks located at the left and upper sides of the macroblock. The present invention is an invention for eliminating the error of the MPM. Therefore, it is possible not to discriminate the boundary between the image of the current image and the adjacent image disposed on the lower and right sides of the current image. That is, the interface between the images may be determined only at the left end and the upper end of the current image. Here, the current image is an image corresponding to a slice output from the parameter and header generation unit 200, and corresponds to an input image corresponding to a bit stream for which the current MPM is to be corrected.

The boundary macroblock extracting unit 315 extracts macroblocks arranged on the boundary between the images in the current image. The boundary macroblock extracting unit 315 determines the size and position of each slice through the parsing of the PPS, and if the determined position is a boundary between the images, the macroblock included in the slice is decoded and extracted . In the present invention, macroblocks arranged at the boundary between images in the current image are referred to as boundary macroblocks. As described above, MPM calculation refers to prediction modes of macroblocks arranged on the left and upper sides of a macroblock. Therefore, even when the macroblocks arranged on the right and the lower end of the current image are arranged on the interface between the images It is not necessary to extract the boundary macroblock.

The neighboring macroblock extracting unit 317 decodes and extracts a macroblock included in the boundary between the image and the adjacent image in the composite image. That is, the macroblock extracted by the boundary macroblock extracting unit 315 from the image arranged adjacent to the current image is decoded and extracted. Hereinafter, the neighboring macroblock extracted by the neighboring macroblock extracting unit 317 will be referred to as a neighboring macroblock for convenience of explanation. That is, in the present invention, the neighboring macroblocks are macroblocks of other images arranged adjacent to the boundary of the current image. Since the MPM is not calculated for the right and bottom boundaries of the current image, it is not necessary to extract the matrices blocks arranged at the upper boundaries and the left boundaries of other images even if they are arranged at the boundaries between the images.

The prediction mode analyzing unit 320 analyzes the prediction modes set for the macroblocks arranged at the boundary between the extracted current image and the neighboring images and the neighboring macroblocks of the neighboring images. The prediction mode analyzing unit 320 includes a current block prediction mode analyzing unit 321 and a neighboring block prediction mode analyzing unit 323. The current block prediction mode analyzing unit 321 analyzes the prediction mode set in the boundary macroblock extracted by the boundary macroblock extracting unit 315 and the neighboring block prediction mode analyzing unit 323 analyzes the neighboring macroblock extracted by the neighboring macroblock extracting unit 317, And analyzes the prediction mode set in the neighboring macroblock extracted from the neighboring macroblock. And transmits the analyzed prediction mode to the MPM regenerator 330. [

The MPM regenerator 330 regenerates the MPM of the boundary macroblock considering the prediction mode of the analyzed neighboring macroblock.

The MPM regenerating unit 330 selects a prediction mode having a lower number among the prediction modes of the neighboring macroblocks analyzed by the neighboring block prediction mode analyzing unit 323 and analyzes the selected prediction mode and the current block prediction mode analyzing unit 321 And determines whether the prediction mode of the boundary macroblock is the same prediction mode.

If the selected prediction mode and the prediction mode of the boundary macroblock are the same, only one flag bit is set to regenerate the MPM. If the selected prediction mode and the prediction mode of the boundary macroblock are different from each other, The MPM is regenerated by setting 3 additional bits including prediction mode information. Here, the number of bits of the flag bit and the additional bit may be changed according to the setting of the image encoding apparatus and the decoding apparatus.

The predicted mode re-encoding unit 340 entropy-encodes the reproduced MPM and transmits the entropy-encoded MPM to the stream combining unit 400.

That is, the image synthesis system and the image synthesizing apparatus according to the present invention generate a synthesized image by modifying the SPS and PPS parameters of the input image and the header of the slice from a plurality of input images applied to the bitstreams BS1 to BSn, Some macroblocks arranged on the boundary are decoded, and MPMs of the macroblocks are generated, encoded, and inserted into the synthesized image. Therefore, it is possible to provide a composite image in which the MPM error is removed by adding only a simple process to the conventional technique of modifying only the SPS and PPS parameters of the input image and the slice header.

6 illustrates an image synthesis method according to an embodiment of the present invention.

Referring to FIG. 3 to FIG. 5, the image combining method of FIG. 6 will be described. First, a plurality of image composing devices VC receive bit streams BS1 to BSn for a plurality of input images (S10). The stream parsing unit 100 of the video synthesizer VC may receive a plurality of bit streams BS1 to BSn in NAL units.

Then, the stream parsing unit 100 analyzes the syntax of each of the plurality of received bitstreams (S20). The stream parsing unit 100 first analyzes the NAL header syntax of a plurality of bit streams BS1 to BSn received in units of NALs to determine a data type, and determines parameters of SPS, PPS, The header is parsed to determine the size of the input image, the size and position where each slice is to be placed, and the location of the macroblock.

Then, the parameter and header generation unit 200 generates a parameter and slice header for the composite image (S30). The parameter and header generation unit 200 merges the SPS and PPS parameters analyzed in the input bitstreams BS1 to BSn and the header of the slice as they are. However, the SPS and PPS parameters and some of the header of the slice header Is changed or newly set to fit the configuration information of the preset composite image.

In order to remove the MPM error, the boundary block extracting unit 310 of the MPM correcting unit 300 analyzes the arrangement of each of the plurality of input images included in the synthesized image (S40). When the arrangement of each input image is analyzed, a boundary between images generated according to the arrangement of each of the plurality of input images is discriminated (S50). When the boundary between the images is determined, the boundary macroblocks arranged at the boundary between the current image and the other images and the neighboring macroblocks adjacent to the boundary macroblock are decoded and extracted (S60).

 The prediction mode analyzing unit 320 of the MPM correcting unit 300 analyzes and determines a prediction mode set in a macroblock located at a boundary between the extracted current image and a neighboring image and a neighboring macroblock located nearby S70).

Thereafter, the MPM regenerating unit 330 selects a prediction mode of a lower number among the prediction modes of the neighboring macroblocks, and compares the selected prediction mode with the prediction mode of the boundary macroblock to regenerate the MPM (S80).

The prediction mode re-encoding unit 340 entropy-codes the regenerated MPM and outputs the entropy-encoded MPM to the stream synthesizing unit 400 (S90).

The stream synthesizing unit 400 inserts parameters and a correction syntax generated by the header generating unit 200 in a plurality of bit streams BS1 to BSn received by the image synthesizing unit VC and the MPM correcting unit 300 The re-coded MPM is inserted to output the composite video (CVBS) as a bit stream (S100).

The method according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and a carrier wave (for example, transmission via the Internet). The computer-readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art.

Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (17)

A stream parsing unit for receiving a bitstream for a plurality of input images and performing a parsing for each of the received plurality of bitstreams;
A parameter and a header generating unit for modifying the syntax analyzed by the stream parsing unit according to configuration information for a predetermined composite image to generate a modification syntax;
A plurality of boundary macroblocks which are macroblocks arranged at a boundary of the input image from each of the plurality of bitstreams and a neighboring macroblock which is the macroblock of another input image arranged adjacent to each of the plurality of macroblocks are decoded An MPM correcting unit for MPM (Most Probable Mode) according to a prediction mode of the neighboring macroblock by analyzing a prediction mode set in each of the boundary macroblock and the neighboring macroblock; And
Replacing an existing syntax included in the bitstream, inserting the MPM generated and regenerated and encoded in the header and the parameter generation unit to generate a bitstream for the composite image including the plurality of input images, A stream synthesizer for outputting a stream; Lt; / RTI >
The MPM correction unit
A boundary block extracting unit for discriminating the boundary of the image according to the arrangement position in which the plurality of input images are arranged in the composite image, and decoding and extracting the boundary macroblock and the surrounding macroblock from the determined boundary;
A prediction mode analyzer for analyzing and discriminating the prediction mode set in each of the boundary macroblock and the neighboring macroblock;
Determining whether the MPM prediction mode of the boundary macroblock is identical to the recalculated MPM prediction mode using the prediction mode set in the neighboring macroblock, part; And
A prediction mode re-encoding unit for encoding the regenerated MPM so that the regenerated MPM can be inserted into a bit stream for the synthesized image; And an image synthesizer for synthesizing the image. delete The apparatus of claim 1, wherein the boundary block extractor
An image layout analyzer for determining a layout position of each of the plurality of input images in the composite image according to the parameter and the configuration information for the composite image set in the header generation unit;
An image boundary determining unit for determining an interface between adjacent images generated according to a position where a plurality of input images are arranged in the composite image;
A boundary macroblock extracting unit which decodes and extracts the boundary macroblocks arranged at the boundary between the images in the current image for correcting the MPM among the plurality of input images; And
A neighboring macroblock extracting unit that decodes and extracts the neighboring macroblocks adjacent to the boundary macroblocks in images arranged adjacent to the current image; And an image synthesizer for synthesizing the image. 4. The apparatus of claim 3, wherein the prediction mode analyzer
A current block prediction mode analyzer for analyzing and determining the prediction mode set in the boundary macroblock; And
A neighboring block prediction mode analyzer for analyzing and determining the prediction mode set in the neighboring macroblock; And an image synthesizer for synthesizing the image. 4. The apparatus of claim 3, wherein the MPM regeneration unit
The MPM prediction mode is determined by discriminating the lowest prediction mode among the prediction modes of the neighboring macroblocks, and if the prediction mode of the MPM prediction mode is the same as the prediction mode of the boundary macroblock, only the flag bit is set And sets an additional bit for designating a prediction mode of the boundary macroblock together with the flag bit if the MPM prediction mode and the prediction mode of the boundary macroblock are different from each other. 6. The apparatus of claim 5, wherein the MPM regeneration unit
Wherein the prediction mode selection unit selects the prediction mode of the MPM prediction mode having the lowest number among the prediction modes of the neighboring macroblocks arranged adjacent to the left and top of the boundary macroblock. The apparatus of claim 1, wherein the parameter and header generation unit
Wherein the correction syntax is generated by modifying SPS, PPS parameters and slice header of the syntax of the plurality of bitstreams. The apparatus of claim 1, wherein the parameter and header generation unit
And the configuration information for the composite image is received and set from outside. A plurality of encoding apparatuses for receiving different original images, encoding them in a predetermined encoding scheme, and outputting bitstreams; And
A plurality of input images are received as the plurality of bit streams, and a syntax for each of the plurality of bit streams is modified according to configuration information for a predetermined composite image to generate a correction syntax, A plurality of boundary macroblocks, which are macroblocks arranged at a boundary of the input image from each of the plurality of bitstreams, and a plurality of boundary macroblocks adjacent to each of the plurality of macroblocks, And decoding the extracted neighboring macroblocks by analyzing a prediction mode set in each of the extracted boundary macroblocks and the neighboring macroblocks and analyzing the prediction modes of the MPMs (Most Probable Mode) is reproduced, re-encoded, Video synthesizer that is inserted into the bitstream for the image; Lt; / RTI >
The image synthesizing apparatus
A stream parser for receiving the plurality of bitstreams and performing parsing;
A header and a parameter for modifying the syntax analyzed by the stream parsing unit to generate the modification syntax;
A plurality of boundary macroblocks which are macroblocks arranged at a boundary of the input image from each of the plurality of bitstreams and a neighboring macroblock which is the macroblock of another input image arranged adjacent to each of the plurality of macroblocks are decoded An MPM correcting unit for correcting the MPM (Most Probable Mode) according to the prediction mode of the neighboring macroblock by analyzing the prediction mode set in each of the boundary macroblock and the neighboring macroblock; And
Wherein the modifying means modifies the existing syntax included in the bitstream by inserting the modified syntax and the re-encoded MPM generated in the header generating unit into the bitstream of the synthesized image, A stream synthesizer for removing an MPM error in a bitstream of a synthesized image; Wherein the image synthesis system comprises: delete 10. The system of claim 9, wherein the image synthesis system
A plurality of image input means for generating the original image and transmitting the same to a corresponding one of the plurality of encoding apparatuses; Further comprising the steps of: 10. The system of claim 9, wherein the image synthesis system
At least one recording medium storing the plurality of original images and transmitting the same to a corresponding one of the plurality of encoding apparatuses; Further comprising: an image synthesizer for synthesizing the image data. A video synthesis method of a video synthesis apparatus of a video synthesis apparatus including a stream parsing unit, a parameter and a header generating unit, an MPM correcting unit, and a stream synthesizing unit,
Receiving the plurality of bit streams for each of the plurality of input images and analyzing the syntax for each of the received bit streams;
Generating a modified syntax by modifying the parsed and parsed header according to configuration information for a predetermined composite image;
Wherein the MPM correcting unit comprises: a plurality of boundary macroblocks, which are macroblocks arranged at the boundaries of the input image from each of the plurality of bitstreams; and a boundary macroblock which is a macroblock of another input image arranged adjacent to each of the plurality of macroblocks, Decoding and extracting the block;
Analyzing a prediction mode set in each of the boundary macroblock and the neighboring macroblock from which the MPM correction unit has extracted and regenerating and re-encoding an MPM (Most Probable Mode) according to a prediction mode of the neighboring macroblock; And
Wherein the stream combining unit replaces an existing syntax included in the bitstream, inserting the modified syntax generated by the parameter and header generating unit and the encoded MPM to generate a bitstream for the composite image including the plurality of input images, Outputting; Lt; / RTI >
The step of decoding and extracting the macroblock
Determining placement positions of the plurality of input images in the composite image according to the configuration information on the composite image;
Determining a boundary between adjacent images generated according to a position of the plurality of input images in the composite image;
Decoding and extracting the boundary macroblocks arranged on the boundary between the images in the current image for correcting the MPM among the plurality of input images; And
Decoding and extracting the neighboring macroblocks adjacent to the boundary macroblocks from images arranged adjacent to the current image; Wherein the image synthesis method comprises: delete 14. The method of claim 13, wherein the step of re-
Analyzing and determining the prediction mode set in the boundary macroblock;
Analyzing and determining the prediction mode set in the neighboring macroblock;
Determining an MPM prediction mode by determining a prediction mode having a lowest assigned number among prediction modes of the neighboring macroblocks;
Reproducing the MPM by determining whether the prediction mode of the boundary macroblock and the determined MPM prediction mode are identical; And
Encoding the regenerated MPM so that the regenerated MPM can be embedded in the bitstream of the synthesized image; Wherein the image synthesis method comprises: 16. The method of claim 15, wherein the step of determining the MPM prediction mode comprises:
Wherein the pre-assigned number of the prediction modes of the neighboring macroblocks arranged adjacent to the left and the upper end of the boundary macroblock is selected as the MPM prediction mode having the lowest prediction mode. 17. The method of claim 16, wherein regenerating the MPM comprises:
If the MPM prediction mode and the boundary macroblock have the same prediction mode, only the flag bit is set, and if the MPM prediction mode and the boundary macroblock have different prediction modes, And setting an additional bit for designating a mode.

Images