KR100554632B1 - Shape encoding method and device for providing real-time object-based video service - Google Patents

Abstract

The present invention relates to a method and apparatus for shape information encoding for providing a real-time object-based service, and more particularly, shape information in real time by reducing the complexity of shape information encoding used in MPEG-4. The present invention relates to a shape information encoding method and apparatus configured to enable encoding of. In the shape encoding method for providing a real-time object-based service according to the preferred embodiment of the present invention, the shape encoding method may include: setting a minimum rectangular boundary (TRB) including an image object of a target; Partitioning the VOP formed according to the shape of the object into a macroblock having M × N pixels (where M and N are natural numbers) and moving the grid starting point of the partitioned macroblock in the partitioning step The method may include encoding in units of macroblocks without performing a process of searching for an information reduction position.

Real time, Encoding, VOPF, MEMC, Object, MPEG, VIDEO, SHAPE

Description

Shape encoding method and device for providing real-time object-based video service             

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram illustrating an algorithm tool (MPEG-4 OVC encoding tools) according to an embodiment of the present invention.

2 is a flowchart illustrating a shape signal encoding procedure according to an embodiment of the present invention.

3 is a diagram showing the overall configuration of a real-time shape signal encoding system according to an embodiment of the present invention.

4 is a diagram illustrating a configuration of a real-time shape signal encoder according to an embodiment of the present invention.

5 is a view showing the operation principle of the real-time VOP forming unit according to an embodiment of the present invention.

6 is a diagram illustrating an encoding method including a motion estimation method according to an exemplary embodiment of the present invention.

Figure 7 illustrates one embodiment of a test image used in the present invention.

8 is a graph showing a result of comprehensively comparing a conventional method, a real-time VOP formation method, a real-time motion estimation method, and a combination of the two methods.

9A to 9C are graphs comparing performances of a real-time VOP forming method, a real-time motion estimation method, and a real-time coding method in which the two methods are simultaneously applied according to the present invention.

<Description of the symbols for the main parts of the drawings>

310: real-time VOP forming unit

320: real-time VOP encoder

330: Multiplexer

321: real-time motion estimation unit

600: MVP candidate

610: MV for shape

620: MV for texture

The present invention relates to a method and apparatus for shape information encoding for providing a real-time object-based service, and more particularly, shape information in real time by reducing the complexity of shape information encoding used in MPEG-4. The present invention relates to a shape information encoding method and apparatus configured to enable encoding of.

MPEG-4 has an advantage of content-based coding or object-based video coding (hereinafter referred to as OVC) based on an understanding of the content of an image. In this OVC, video contents are divided into objects and backgrounds, processed, and transmitted. Therefore, various types of operations and displays are possible according to the user's intention. Can be manipulated more actively according to their preferences to create an environment suitable for them.

According to the first edition of MPEG-4's visual part, MPEG-4 visual has 19 profiles, but MPEG-4, which is actually serviced, is not OVC, but frame-based video. coding). That is, among the 19 profiles, a profile widely used as a mobile or internet application is an FVC-based simple profile designed by a need of a low bit rate application. At present, the number of commercially available MPEG-4 OVC applications is not large, and there is a lack of research on low-rate applications that can be serviced within the current infrastructure.

Although frame-based MPEG-4 video has been actively serviced since 2000, one of the key problems in which real-time object-based services are not commercialized is the complexity of object-based video encoding compared to frame-based. Video represented as an Arbitrary shape is a motion information (motion or motion, hereinafter 'motion' or 'motion') and texture (texture or object content, hereinafter referred to as 'texture' or 'texture') information in the bitstream In addition, it has shape information. This adds additional complexity to the encoder side as well as to the decoder.

As described above, the MPEG-4 OVC according to the prior art has the following problems.

First, according to the prior art, in the limited real-time arbitrarily shaped object generation part, the boundary of each video object has to be set every frame. Here, the blue screen technique using chroma keying mainly uses the shape separation method, but this is only available in limited fields.

Second, the prior art has a problem that the efficiency is lowered due to the increased complexity due to shape coding. The FVC-based coding scheme consists of two parts, motion and texture. In general, the encoding procedure is more expensive and time consuming than the decoding procedure, and among these encoding procedures, the motion estimation / motion compensation (MEMC) is more time consuming and expensive. . Due to such a MEMC, real-time encoding including the MEMC has a difficult problem. Here, in addition to the coding of the FVC, the OVC adds a complexity called shape coding, and there is a problem in that efficiency decreases due to the complexity of the coding.

In particular, the proportion of motion estimation of shape coding in the MPEG-4 object-based video encoding process is 75% or more of the total compression time. However, according to the prior art, there have been several studies to optimize an encoder that requires more complicated processing than a decoder, but various techniques have been tried to reduce the complexity of MEMC for texture. However, research on shape MEMC is insufficient.

Accordingly, the present invention has been made to solve the above problems, and is a coding method capable of encoding shape information in real time by reducing the complexity of SHAPE INFORMATION ENCODING among a plurality of coding tools used in MPEG-4. And providing an apparatus. In other words, the present invention can provide an effective encoding technique that can reduce the complexity of an encoder by analyzing a tool currently used in MPEG-4 video encoding and a MEMC algorithm in shape encoding that has not been studied before.

Another object of the present invention is to provide a method and apparatus for encoding shape information that can be encoded in real time without substantially reducing performance by omitting an algorithm previously considered essential for the efficiency of encoding. That is, the present invention provides an encoding method and apparatus that can fundamentally solve the complexity of encoding by not adding a separate algorithm process for increasing the efficiency of encoding to the method used for VOP formation and MEMC part. Is in. In particular, the present invention saves execution speed more effectively than any conventional method by omitting the recursive spiral search itself.

In addition, another object of the present invention is to propose a problem and a solution to the complicated processing of the Intelligent VOP Formation (IVOPF) algorithm in the VOP forming step of setting the VOP position.

According to a first aspect of the present invention, a shape encoding method for providing a real-time object-based service may be provided.

According to a preferred embodiment, the shape encoding method may include setting a minimum rectangular boundary (TRB) including an image object of an object, and using the minimum rectangular boundary, a VOP formed according to the shape of the object. Is divided into a macroblock having pixels of M × N (where M and N are natural numbers) and the process of finding a position of decreasing image information amount of an object while moving the grid starting point of the partitioned macroblock in the partitioning step. In addition, the method may include encoding the macroblock unit.

Here, the method may further include determining whether the encoding scheme is real time. The method may further include setting an MV of a current binary alpha block (BAB) as MVP without performing an algorithm including any block comparison in setting a candidate group for motion estimation. have.                     

According to another preferred embodiment, the shape coding method for providing a real-time object-based service is to form a VOP for shape coding, VOP formed according to the shape of the object as a macroblock having M × N pixels In the step of partitioning (where M and N are natural numbers) and setting a candidate group for motion estimation in units of the macroblock, the current binary alpha block (BAB) is not performed without performing an algorithm including any block comparison. The MV of) may be set to MVP.

In order to achieve the above object, according to the second aspect of the present invention, a shape encoding apparatus for providing a real-time object-based service may be provided.

According to a preferred embodiment, the shape encoding apparatus for providing a real-time object-based service sets the minimum rectangular boundary (TRB) including the image object of the object, and using the minimum rectangular boundary, The VOP is partitioned into macroblocks having M × N pixels, and the search for the information reduction position of the image of the object is performed while moving the grid starting point of the partitioned macroblock in the partitioning step. Instead, the VOP forming unit for encoding the macroblock unit (where M and N are natural numbers), the shape encoder for encoding the VOP formed in the VOP forming unit, and the encoded shape signal are multiplexed into a bitstream. It may include a multiplexer for transmission.

Here, the shape encoding apparatus may determine whether the encoding scheme is real time. Further, in setting a candidate group for motion estimation, the apparatus does not perform an algorithm including any block comparison, and further comprises a real-time motion estimator for setting an MV of a current binary alpha block (BAB) as an MVP. It may include.

According to another exemplary embodiment, a shape encoding apparatus for providing a real-time object-based service forms a VOP for shape coding, and divides the VOP formed according to the shape of the object into a macroblock having pixels of M × N. A multiplexer for multiplexing the encoded shape signal, a multiplexer for transmitting the encoded shape signal, and a candidate group for motion estimation in units of macroblocks; Therefore, a motion estimation unit for setting an MV of a current binary alpha block (BAB) as an MVP may be included without performing an algorithm including any block comparison.

Hereinafter, exemplary embodiments of a shape encoding method for providing a real-time object-based service according to the present invention will be described in detail with reference to the accompanying drawings. The components to be given the same reference numerals and duplicate description thereof will be omitted.

Compression Algorithm Tool and Object-Based Coding Flowchart

Intelligent VOP formation algorithm and motion estimation algorithm including IVOPF which is executed to increase the compression rate among existing coding tools require much processing time and become a lot of obstacle in real time coding. Most of the previous studies on coding algorithms including VOP formation and motion estimation suggest new algorithms to increase the compression rate or to reduce the execution speed. However, according to the prior art, it is not possible to fundamentally solve the spiral search problem which complicates the calculation of the motion vector (MV). Hereinafter, various encoding algorithm tools will be introduced in FIG. 1, and a procedure of an encoding algorithm according to the present invention will be described in FIG. 2.

1 is a diagram summarizing an algorithm tool (MPEG-4 OVC encoding tools) according to an embodiment of the present invention.

Referring to FIG. 1, a general encoding algorithm used in a process of encoding a binary shape signal is summarized, and a check mark is placed on a portion which takes a lot of time. The present invention can provide a method and apparatus capable of encoding without significant performance degradation in the case of a real-time service without using an existing complex algorithm. Hereinafter, various algorithm tools of the object-based service, for example, MPEG-4 OVC, MPEG-4 OVC decoding process, MPEG-4 OVC encoding process, VOP Formation, Binary shapes Shape coding and the like will be briefly described.

The shape coding procedure may be divided into a binary shape coding 140 and a VOP formation 100.

In binary shape coding 140, an alpha map with an 8-bit value indicates the transparency of a given image. If the pixel value is 0, it becomes a transparent pixel. If the value is 255, it becomes an opaque pixel. If the pixel value is between 0 and 255, the pixel overlaps with the background pixel value. Here, the binary shape may have a value of 0 and 255 as an alpha value, and since the binary shape data represents the opacity of the pixel rather than the color information, the relationship between the binary shape and the corresponding texture data Is not as close as motion and texture. If the alpha map data has a value between 1 and 255, it is referred to as a grayscale shape coding 150 image. The binary shape coding 140 may include mode decision, motion estimation / compensation (MEMC), subsampling, scan order decision, inter / intra CAE, and the like. It may include a scalable shape.

The VOP Formation 100 refers to an algorithm for extracting and forming an object. Hereinafter, each frame of the video object will be referred to as a video object plane (VOP). If the video object is arbitrarily shaped video rather than rectangular, each Binary Alpha Block (BAB) in the shape part has an all opaque with a total pixel value of 255, a partial opaque with a partial pixel value of 255, and a total pixel value of zero. It is divided into all transparent. Here, instead of writing the entire frame to generate and refer to the BAB, one may use the smallest rectangular boundary (hereinafter referred to as TRB) containing the shape to be extracted.

Here, since the size of the TRB is often not a multiple of 16, some macroblocks have unnecessary transparent pixels. Conventional coding algorithms including intelligent VOP formation (IVOPF) include most of the process of calculating the starting pixel value of the first macroblock of a video object in order to increase the compression rate. The VOP formation algorithm according to the present invention proposes an encoding method without deterioration in performance even without including an algorithm for reducing unnecessary transparent pixels.

As shown in Table 2 (BAB Coding Modes), there are seven modes of shape coding, and a mode decision 110 refers to a procedure for determining such a mode.

Type Number Coding mode Usage 0 MVDs == 0 && no update P- or B-VOPs One MVDs! = 0 && no update P- or B-VOPs 2 All transparent All VOPs 3 All opaque All VOPs 4 intraCAE All VOPs 5 MVDs == 0 && interCAE P- or B-VOPs 6 MVDs! = 0 && interCAE P- or B-VOPs

In Table 1, in case of 0, 2, and 3 modes, no further encoding is required, and IntraCAE requires an intra-mode CAE process. no update copies the BAB of the previous VOP and executes the MC. InterCAE executes the MC process and inter-mode CAE, and MVD means the difference between the current motion vector and the referenced motion vector.

The motion estimation / motion compensation (MEMC) algorithm 120 is an efficient algorithm, especially when the video object has little motion or moves slowly. The motion estimation / motion compensation (MEMC) algorithm 110 includes a process of determining a motion vector predictor (MVP), a motion estimation process, and a motion compensation process.

2 is a flowchart illustrating a shape signal encoding procedure according to an embodiment of the present invention. In order to avoid the confusing explanation, the procedure of the basic encoding process is omitted in the flowchart of FIG. 2 and only the encoding process characteristic of the present invention is illustrated.

MPEG-4 OVC codes and steps are divided into shape coding, texture coding and motion estimation and compensation, and are generally based on 16 * 16 pixel MB. based coding. As shown in Table 1, the encoding process has many encoding tools, and due to algorithms that require a lot of time, it is difficult to perform real-time encoding. The present invention can propose a fundamental solution proposal for VOP formation and shape motion estimation / compensation which takes a lot of time in relation to shape coding.

Hereinafter, an encoding method for a real-time object based service according to the present invention will be described with reference to FIG. 2.

First, it is determined whether or not the VOP forming step in step S200, and in the case of the VOP forming step, in step S210, it is determined whether the service is a real-time service. As a result of the determination, if it is not a real time service, a VOP is formed according to a conventional method (for example, IVOPF) in step S220, and in case of a real time service, a VOP is formed according to the real time VOP formation method according to the present invention in step S230. do. Real-time VOP formation method according to a preferred embodiment of the present invention is to form a VOP using only TRB (see Fig. 5)

The present invention verifies the practicality of the VOP forming portion according to the prior art by analyzing the complexity calculation and the compression efficiency, thereby selectively using a real-time VOP forming algorithm with no significant difference in performance even if it does not include the minimum non-transparent block. Thus, a real time encoding service can be provided. Here, of course, it may be set to encode using TRB regardless of the real-time service.

Thereafter, in step S240, it is determined whether the step of the encoding step is a shape motion estimation / motion compensation (MEMC) step. As a result of the determination, if it is not a shape motion estimation / motion compensation (MEMC) step, a motion estimation / motion compensation (MEMC) procedure of performing a block comparison according to the conventional method may be performed in step S260. If it is determined that the motion estimation / motion compensation (MEMC) step is performed, it may be configured in step S250 to determine whether the service is a real-time service.

As a result of the determination, if it is not a real time service, a motion estimation / motion compensation (MEMC) algorithm according to the conventional method is performed in step S260, and if it is a real time service, real time motion estimation / motion compensation according to the present invention in step S270 Perform the algorithm.

Motion estimation / motion compensation (MEMC) can be used in both shapes and textures. This MEMC is complicated and time consuming during the encoding process. Previous research has focused on solving the complexity of MEMC in texture coding, and similar algorithms have been used for shapes. According to the present invention, in the motion estimation / motion compensation (MEMC) step in shape coding, by providing a real-time motion estimation / motion compensation (MEMC) algorithm in which a block comparison procedure is omitted, a real-time object-based service is provided without degrading performance. (See FIG. 6).

According to the present invention, in the real-time service, the motion vector difference (MVD) may be always set to 0 by simplifying the MEMC process for the shape. Therefore, in Table 2 in which various modes of BAB Coding are shown, Modes 1 and 6 do not occur.

Diagram of Real-time Object-based System

Hereinafter, the configurations of the real-time VOP forming unit and the real-time VOP encoder according to the present invention will be described in detail with reference to the accompanying drawings. In FIG. 3, the configuration of the entire system including the real-time VOP forming unit will be described. In FIG. 4, the detailed configuration of the real-time encoder will be described.

3 is a diagram showing the overall configuration of a real-time shape signal encoding system according to an embodiment of the present invention.

MPEG-4 OVC classifies a scene into a background shape and an object shape, and additional information, called a shape or alpha map, is encoded in a bitstream. Shape information is used to distinguish whether a given pixel is a transparent (part outside the shape) or an opaque (part inside the shape). This shape information will be referred to as a binary shape.

Hereinafter, a description will be given of an encoder 300 that performs a shape information encoding system and a decoder 350 that performs decoding according to the present invention.

First, the encoder 300 of the MPEG-4 OVC includes a real time VOP forming unit 310, a real time VOP encoder 320, and a multiplexer 330.

In this case, when a real time VOP forming unit 310 receives an image sequence (SEQUENCE) to be transmitted or stored, it is divided into object image units to form different VOPs. The real-time VOP formation unit 310 according to the present invention is configured to extract the minimum TRB without going through a high complexity process in the case of real-time transmission (see FIG. 5). That is, the present invention is configured to extract only the content object by a preset method (for example, TRB) without performing an optimization algorithm performed to increase the compression ratio as in the conventional method. According to the conventional method, it was limited to increase the compression efficiency while maintaining non-transparent MB at the minimum or proper level.

Meanwhile, each VOP formed by the real-time VOP forming unit 310 is input to the VOP encoders 320-1, 320-2, ..., respectively, encoded for each VOP, multiplexed by the multiplexer 330, and then bit stream (BIT). STREAM).

The structure of the decoder 350 is as follows. The coded signal of the VOP, which is coded through the encoder 300 and is transmitted in the bitstream, is separated for each VOP in the demultiplexer 360. Each of the separated VOP coded signals is decoded by the decoder 370, which may include a plurality of decoders 370-1, 370-2, 370-3,... And the decoder 370. The decoded signal output from) is synthesized by the combining unit 23 and output as an original image. As such, the VOP extracted by the real-time VOP forming unit may be encoded through the real-time encoder described below.

4 is a diagram illustrating a configuration of a real-time shape signal encoder according to an exemplary embodiment of the present invention.

The configuration of the real-time VOP encoder will be described with reference to FIG. 4. When a VOP for each target image formed by the real-time VOP forming unit 310 is input to the motion estimation unit 321, the real-time motion estimation is performed. The government 321 may estimate the motion of the macroblock unit from the authorized VOP. The motion information estimated by the real-time motion estimation unit 321 is input to a motion compensation unit 322 to compensate for the motion.

Each MB of the VOP whose motion is compensated by the motion compensator 322 is input to the subtractor 323 together with each MB of the referenced VOP formed by the VOP forming unit 310 to detect a difference value. The difference value detected at 323 is input to the texture encoder 324 to encode texture information of the object in sub-block units of the macroblock. Meanwhile, the VOP whose motion is compensated by the motion compensator 322 and the texture information of the object encoded by the texture encoder 324 for encoding the object texture information are inputted and added by the adder 325. The output signal of the adder 325 is input to a PREVIOUS RECONSTRUCTED VOP 326 to detect a previous VOP corresponding to the VOP of the image immediately before the current image. The previous VOP detected by the previous VOP detector 326 is input to the real time motion estimator 321 and the motion compensator 322 and used for motion estimation and motion compensation.

The VOP formed by the real-time VOP forming unit 310 is input to the real-time shape encoder 327 to encode shape information. The output signal of the shape encoder 327 may be input to the motion estimator 321, the motion compensator 322, and the texture encoder 324 to be used for encoding motion estimation, motion compensation, and internal information of an object. . In addition, motion information estimated by the real-time motion estimation unit 321, object texture information encoded by the texture encoder 324, and shape information encoded by the shape encoder 327 may be applied to the multiplexer 330. After the multiplexing, the data is output to the multiplexer 330 through the buffer 340 and transmitted in a bitstream.

In the MPEG-4, a technique applied to the shape encoder 320 for encoding each VOP transmitted from the real-time VOP forming unit 310 includes N × N blocks (N = 16, 8, 4). MMR shape information coding technology (MMR shape coding TECHNIQUE) for encoding shape information based on the shape information, vertex-based shape information coding technology (VERTEX-BASED shape coding) TECHNIQUE), BASELINE-BASED shape coding TECHNIQUE, and CONTEXT-BASED ARITHMETIC CODING, but these conventional techniques basically provide motion estimation. In doing so, an iterative block comparison process is required.

The real-time motion estimation unit 321 according to the present invention is configured to set a predetermined MVP as a current MV without performing such block comparison association when encoding shape information in the case of a real-time service.

Real-time VOP formation method and coding method using real-time motion estimation

Hereinafter, a method for forming a real time VOP and a method for encoding a real time VOP according to the present invention will be described in more detail with reference to FIGS. 5 and 6. A method of forming a real-time VOP is described in FIG. 5, and a method of encoding a real-time VOP is described in FIG. 6.

5 is a view showing the operation principle of the real-time VOP forming unit according to an embodiment of the present invention.

In FIG. 5, in order to increase compression efficiency, an IVOPF according to an embodiment and a real-time VOP forming method according to the present invention will be described with reference to various VOP forming methods including the existing optimization process.

IVOP includes an optimization algorithm that finds the location of the VOP that contains the least number of non-transparent MBs on the assumption that a small number of non-transparent MBs can minimize motion and texture coding and lead to better compression. Referring to Table 2 below, various VOP location setting methods are illustrated.

Selection Mode Semantics IVOPF Find the minimum number of nontransparent MBs. 8x8 Bnd (8x8) Find the minimum number of 8x8 boundary blocks Worst Find the maximum number of nontransparent MBs Trans / Bnd Min. (TBM) Find the minimum number of transparent and boundary MBs. All Min (AM) Find the minimum number of MBs (or try to find the minimal size of VOP) Tightest Rectangle (TR) Do not use IVOPF at all.

Referring to Table 3 below, the results of experimenting with the above-described VOP forming method using a predetermined test image (see FIG. 7) are shown.

Test sequence IVOPF 8x8 Worst TBM AM TR akiyo 105.3 142.9 111.1 144.4 105.3 93.7 coastguard 169.3 170.7 169.9 170.6 169.3 168.8 container 122.5 123.1 122.7 123.4 122.7 121.8 dancer1 73.4 88.0 80.9 86.8 74.9 74.4 dancer2 120.4 142.3 127.9 141.5 121.8 118.1 foreman 118.4 127.9 120.1 128.0 118.5 116.3 news 141.2 144.8 141.9 145.5 137.2 138.2 saxophone 200.6 243.0 207.8 246.0 203.9 191.3 singer 62.0 79.1 69.4 78.2 63.8 62.1 stefan 65.5 77.1 72.7 76.9 67.1 67.3 bream 120.8 144.9 126.7 143.2 120.7 116.3 child 103.6 135.3 114.4 136.2 106.8 99.3 cyclamen 201.8 211.1 203.1 206.1 207.5 207.3 fishandlogo 127.2 155.3 138.9 155.8 131.7 125.7 weather 76.4 97.6 81.8 98.9 78.1 71.9 Average 120.6 138.9 126.0 138.8 121.9 118.2

Referring to Table 3 above, there is no formation method that provides better results than IVOPF except the real-time VOP formation method (TR) according to the present invention. In addition to the complexity and execution speed, it can be seen that the evaluation of how much the same as the original image shows little difference between the IVOPF and the TR (see FIG. 9A).

Referring to FIG. 5, a VOP Formation principle for determining a binary alpha block (BAB) is illustrated. Here, the horizontal size of the VOP is defined by the width of the VOP, the vertical size is defined by the height of the VOP, the formed VOP is the grid starting point in the upper left corner, M and N pixels in the X axis and Y axis, respectively It will be partitioned into M × N macroblocks with. For example, the X and Y axes may be divided into 16 × 16 macroblocks each having 16 pixels, and M and N may be encoded in units of subblocks by a texture coding unit described later. Each can be set to an even number to make it possible.

First, the VOP formation method according to the conventional method will be described. In general, the TR 510 of a given object 500 does not exactly match a multiple of MB. Therefore, in order to determine the VOP position, the upper left 510 of the TR is designated as the first MB.

In IVOPF, a preferred embodiment of the conventional method, the upper left 510 of the TR is regarded as the lower right of the control MB, and the control MB 520 is determined. In Control MB 520, pixels across one pixel are used to analyze the number of VOPFs and MBs. For example, the total number of pixels used is 64 (8 * 8). The reason why one pixel is used across one pixel is that the video format is YUV (4: 2: 0), and the color (U and V) values are included in four luminance values (Y). Each pixel of the 64 pixels in the control block 520 becomes a position candidate pixel of the first MB. You can then calculate the total number of MBs, all transparent MBs, and the number of non-transparent (all opaque and partially transparent) MBs. After all cases (64 pixels) have been calculated, the conventional technique in which the VOP is formed based on the pixel having the smallest number of non-transparent MBs takes a long time due to the iterative processing. The worst case is to calculate all 64 and compare the number of non-transparent MBs, and this iterative algorithm is one of the factors that makes real-time coding difficult.

According to the real-time VOP formation method according to the present invention, instead of the existing VOP formation algorithm including the iterative processing algorithm, it is configured to form the VOP using only the TR (510). Of course, according to the present invention, in the case of not the real-time service, it is a matter of course that the VOP can be formed using various intelligent VOP formation algorithms. In addition, the real-time VOP forming method according to the present invention can be used regardless of the real-time service. Referring to FIG. 9A, by verifying by analyzing the image quality and the amount of compression for the conventional VOP forming method, it can be seen that there is no significant difference in performance even if the conventional VOP forming process is not included in the shape coding. have.

6 is a diagram illustrating an encoding method including a motion estimation method according to an exemplary embodiment of the present invention.

Binary Shape Coding In MPEG-4, MEMC is used for shape and texture and compares and adds pixel values to calculate the similarity between the current BAB and the referenced BAB. . Because shape blocks compare binary values, MEMCs for shapes are less complex than MEMCs for textures, but there is a common part of block comparison operations. It is specified to use spiral search for the calculation part. The MB candidates that follow from the center point of the expected MVP are obtained through this spiral search. This method can set the pixel that is closest to the center point, but due to the algorithm that iterates through +/- 16 pixels, it is one of the major causes of increasing the complexity of the encoder. Various techniques for reducing the complexity of the MEMC have been proposed, but most of them are concentrated in the texture MEMC, and for the shape MEMC, the texture MEMC is used as an application.

Referring to FIG. 6, an example 600 of a candidate of an MVP for a corresponding BAB is shown. According to an embodiment, the candidate 600 of MVP is divided into MV for shape 610 and MV for texture 620. According to the conventional method, the previous MV of the texture as well as the shape is used to determine the MVP, and only MV for shape is used for shape-only coding. Currently the surrounding MVs of BAB are shown. MVP is set to the first MV that is not 0, and 0 if there is no valid MV. Once the MVP has been determined, if the result of comparing pixel values to calculate the similarity between the current BAB and the referenced BAB is less than or equal to the threshold value, the determined MVP is set as the MV of the corresponding BAB. On the other hand, if there is a difference over a certain value, motion estimation is started by calculating the error between the referenced MB within the MVP center point-/ + 16 pixel range and the current MB. The current search method performs a spiral search on the shape as well as texture motion estimation. The important point is that shape motion estimation is as complex as texture. So texture MEMC only optimization solves the complexity problem of MEMC only half way.

According to the present invention, unlike the MEMC for texture, the MEMC for shape is not included in the optimization algorithm, and thus, no block comparison is performed. For example, the MVP is configured to set and encode the MV of the current BAB. Therefore, the encoder according to the present invention can set the most suitable among 7 coding modes. By eliminating the helical search, the throughput in shape coding is reduced, that is, the motion estimation method according to the conventional method is used in the case of a service other than the real time coding in the motion estimation / motion compensation step, and the block comparison in the real time service. It is configured to use a motion estimation method that does not.

According to the present invention, since there is no difference in image quality even when the motion estimation process is not performed at the time of shape coding, the motion estimation process may not be performed at the time of shape coding. Therefore, according to the present invention, in shape coding, the MV of the current BAB is set to MVP.

Referring to FIG. 9B, by verifying by analyzing the image quality and the amount of compression of the conventional VOP coding method, it can be seen that there is no significant difference in performance even if the motion estimation process is not included in the shape coding.

Performance and speed comparison

Hereinafter, the performance and effects of the shape coding method for the real-time object-based service according to the present invention and the existing shape coding method will be described. Referring to Figure 7, one embodiment of the test image used in the present invention is shown. In FIG. 8, a result of comprehensively comparing a conventional method, a real-time VOP forming method, a real-time motion estimation method, and a combination of the two methods will be described. 9A to 9C, the performance of each scheme will be described in comparison with the conventional scheme.

7 is a view showing an embodiment of a test image used in the present invention. Referring to Figure 7, one embodiment of the test image used in the present invention is shown. In FIG. 7, only a representative test image is illustrated, but experiments were performed using a plurality of test images indicated in Table 4 below.

Test sequence No. of frames Format akiyo 300 CIF coastguard 300 CIF container 300 CIF dancer1 250 CIF dancer2 250 CIF foreman 300 CIF news 300 CIF saxophone 250 CIF singer 250 CIF stefan 300 CIF bream 300 SIF child 300 SIF cyclamen 300 SIF fishandlogo 300 SIF weather 300 SIF

FIG. 8 is a graph showing a result of comprehensive comparison between a conventional method, a real-time VOP formation method, a real-time motion estimation method, and a combination of the two methods.

Hereinafter, with reference to FIG. 8, the present invention can provide a real-time object-based service without compromising performance by reducing the complexity of an encoder and reducing an encoding time, thereby overcoming a time difference with a decoder. Will be described.

FIG. 8 is a graph showing the results of Table 5, which shows the encoding time experimented using the test image shown in Table 7. Here, the number represents the result of the encoding execution time result, Ref SW (Reference Software) 800 refers to the encoding scheme defined in the standard, and the T (Tightest Rectangular) 810 uses the TRB according to the present invention. Refers to an encoding method when a VOP is formed. And NMS (No Motion Search) 820 refers to the coding method using the real-time motion tracking method according to the present invention. Proposed Encoding Conditions (PEC) 830 refers to a real-time shape encoding method using a combination of a real-time VOP formation method and a real-time motion estimation method. The coding speeds for the respective test images are shown, and when the average value 840 is compared, it can be seen that the real time coding method combining the real time VOP formation method and the real time motion estimation method requires the least time.

Test sequence Ref. SW TR NMS PEC akiyo 105.3 93.7 98.1 88.7 coastguard 169.3 168.8 161.0 161.6 container 122.5 121.8 118.8 119.0 dancer1 73.4 74.4 62.4 62.1 dancer2 120.4 118.1 101.9 99.0 foreman 118.4 116.3 107.6 105.9 news 141.2 138.2 137.1 134.6 saxophone 200.6 191.3 177.5 167.4 singer 62.0 62.1 54.5 53.7 stefan 65.5 67.3 55.4 55.9 bream 120.8 116.3 104.8 101.2 child 103.6 99.3 86.8 81.0 cyclamen 201.8 207.3 178.4 179.6 fishandlogo 127.2 125.7 97.9 94.4 weather 76.4 71.9 69.4 64.5 Average 120.6 118.2 107.4 104.6

Referring to FIG. 8 and Table 5, in the case of an encoder using a real-time motion estimation method 820 without a motion search (MS), execution time is shortened by 10.9% compared to the reference software (RF) 800. In particular, fishandlogo images can be up to 23% faster.

According to the present invention, a real-time encoding method 830 that simultaneously applies a real-time VOP formation method 810 and a real-time motion estimation method (no MEMC for shape) 820 using a Tightest Rectangle (TR) has an average coding rate 840. 13.2% improvement over the conventional speed. The most striking effect was the fishandlogo image, which was reduced by 25.7%.

And the results of the experiment in the case of Shape-only coding with reference to Table 6 below.

Test sequence Ref. SW PEC Speedup (%) akiyo 28.6 9.6 66.3 coastguard 19.3 11.6 40.0 container 13.0 9.5 27.3 dancer1 24.5 9.9 59.7 dancer2 38.2 13.5 64.7 foreman 28.2 13.4 52.5 news 15.9 10.3 34.8 saxophone 53.5 16.2 69.7 singer 22.8 9.8 56.9 stefan 22.2 9.1 59.1 bream 33.0 11.6 64.8 child 39.5 12.9 67.3 cyclamen 44.7 20.2 54.7 fishandlogo 54.2 16.6 69.5 weather 22.7 8.4 62.8 Average 30.7 12.2 56.7

Shape-only coding means that only shape information is encoded and decoded. Referring to Table 6, it can be seen how much the complexity is reduced in the shape coding portion. As a result, it can be seen that the average execution speed of the present invention accounts for less than half of the conventional average speed.

Table 7 below summarizes the Comparison of the Number of Instructions according to the present invention.

Test sequence Ref. SW PEC Reduction (%) akiyo 18753 12976 30.8 fishandlogo 19215 15701 18.3

Referring to Table 7, it can be seen that the overall instruction executed is significantly reduced.

9A to 9C illustrate performance comparison graphs comparing a real-time VOP forming method, a real-time motion estimation method, and a real-time encoding method in which the two methods are applied simultaneously with the conventional methods.

Hereinafter, FIG. 9A compares the conventional method with the real-time VOP forming method, FIG. 9B compares the conventional method with the real-time motion estimation method, and FIG. 9C shows the real-time shape coding method combining the conventional method with the real-time VOP formation and real-time motion estimation method. It will be described by comparing. The graph is a result of the experiment based on the (Akiyo sequence) of the test image of FIG.

9A to 9C, the X axis represents the compression ratio and the Y axis represents the noise ratio measured by PSNR. Here, PSNR (Peak Signal to Noise Ratio) means the difference between the original image and the encoded image.

9A to 9C, the PSNRs 920, 940, and 960 of the encoder according to the real-time coding scheme according to the present invention are almost different from the PSNRs 910, 930, and 950 of the encoder. Shows none. Therefore, the present invention can reduce the complexity of the encoder without degrading the overall compression rate and the image quality deterioration, and it can be seen that there is no change in image quality that can be visually recognized by a human.

As described above, the present invention can provide a method that can reduce the complexity of shape coding more than half while preserving the compression efficiency of motion, shape, and texture. According to an embodiment of the present invention, the present invention is a spiral during the processing of the use of the TBT (Tightest Rectangular Boundary) and MEMC for shape rather than IVOPF (Intelligent VOP Formation) to reduce the complexity in shape coding By providing a real-time shape coding method and apparatus in which the search is omitted, it is understood that the complexity of shape coding is reduced by 56%.

Although the technical idea of the present invention has been described in detail according to the above-described embodiment, the above-described embodiment is for the purpose of description and not of limitation, and a person of ordinary skill in the art of the present invention It will be understood that various embodiments are possible within the scope.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and is an encoding method capable of encoding shape information in real time by reducing the complexity of SHAPE INFORMATION ENCODING among a plurality of encoding tools used in MPEG-4, and There is an effect that can provide a device.

In addition, the present invention can fundamentally solve the MEMC problem of MPEG-4 OVC, which is the biggest problem in object-based encoding. In shape coding of MPEG-4 OVC, ME occupies 75% or more of coding execution time. However, the existing research method recognizes this movement estimation as essential and concentrates on improving its efficiency. According to the present invention, unlike the texture encoding, the shape encoding raises the question of the necessity of motion estimation and provides an encoding method that omits it, thereby significantly reducing the encoding execution speed of the object-based service.

In addition, the present invention has the effect of providing a real-time object service without deterioration in performance by using a titanic square (TR) coding method without using an algorithm for increasing compression efficiency when forming a VOP.

The present invention can commercialize real-time object-based services in network environments such as digital broadcasting studios, mobile handsets and Internet PCs.

Although the above has been described with reference to a preferred embodiment of the present invention, those skilled in the art to which the present invention pertains without departing from the spirit and scope of the present invention as set forth in the claims below It will be appreciated that modifications and variations can be made.

Claims (10)

In the shape encoding method for providing a real-time object-based service, Determining whether the encoding is a real-time service; In the case of encoding for a real-time service, setting a minimum rectangular boundary (TRB) including an image object of an object; Partitioning the VOP formed according to the shape of the object into a macroblock having M × N pixels using the minimum rectangular boundary, where M and N are natural numbers; And Encoding the data in units of macroblocks without moving the grid start point of the partitioned macroblocks in the partitioning step and searching for a position of reducing the amount of video information of an object; Shape encoding method of an object-based service comprising a. delete The method of claim 1, In setting a candidate group for motion estimation, without performing an algorithm including any block comparison, setting the MV of the current binary alpha block (BAB) as MVP Shape encoding method of the object-based service, characterized in that it further comprises. In the shape encoding method for providing a real-time object-based service, Determining whether the encoding is a real-time service; In the case of encoding for a real-time service, forming a VOP for shape encoding; Dividing the VOP formed according to the shape of the object into a macroblock having M × N pixels, where M and N are natural numbers; And In setting a candidate group for motion estimation in units of macroblocks, without performing an algorithm including any block comparison, setting an MV of a current binary alpha block (BAB) as an MVP Shape encoding method of an object-based service comprising a. delete In the shape encoding apparatus for providing a real-time object-based service, In the case of encoding for a real-time service, a Tightest Rectangular Boundary (TRB) including an image object of an object is set, and the VOP formed according to the shape of the object by using the minimum rectangle boundary is a pixel of M × N. A VOP forming unit for dividing into macroblocks having a symbol and encoding the macroblock unit without performing a process of searching for a reduced information position of an image of an object while moving a grid start point of the partitioned macroblocks. , N is a natural number); A shape encoder for encoding the VOP formed by the VOP forming unit: and Multiplexer for multiplexing the coded shape signal and transmitting it in a bitstream Shape encoding apparatus of the object-based service comprising a. delete The method of claim 6, In setting a candidate group for motion estimation, a real-time motion estimation unit that sets an MV of a current binary alpha block (BAB) as an MVP without performing an algorithm including any block comparison. Shape encoding apparatus of the object-based service further comprises. In the shape encoding apparatus for providing a real-time object-based service, In the case of encoding for a real-time service, a VOP forming unit for forming a VOP for shape coding, and partitioning the VOP formed according to the shape of the object into a macroblock having M × N pixels; A shape encoder for encoding the VOP formed by the VOP forming unit: A multiplexer for multiplexing the encoded shape signals and transmitting them in a bitstream; and In the case of encoding for a real-time service, in setting a candidate group for motion estimation in units of macroblocks, MVP does not perform an algorithm including any block comparison, but MVP wins the MV of a current binary alpha block (BAB). Motion estimator for setting to Shape encoding apparatus of the object-based service comprising a. delete

Images