KR20180108316A - Parallel encoding method and apparatus - Google Patents

Abstract

Disclosed are an image parallel encoding method and an apparatus thereof capable of overcoming deterioration of image quality. According to one embodiment of the present disclosure, the image encoding method may comprise the processes of: detecting characteristics of at least one picture previously inputted and storing characteristics of the picture in advance in a database; dividing the inputted picture to construct a plurality of divided pictures; detecting characteristics of each of the divided pictures; detecting the previously stored picture corresponding to the detected characteristics of the divided pictures from the database; constructing a boundary predictive picture in which a pixel value of the detected and pre-stored picture is reflected in a boundary area of the divided pictures; and processing parallel prediction for the divided pictures using the boundary prediction picture as a reference picture.

Description

[0001] PARALLEL ENCODING METHOD AND APPARATUS [0002]

This disclosure relates to image encoding techniques, and more particularly, to a method and apparatus for parallel processing of images.

Recently, the demand for high resolution and high quality images such as high definition (HD) image and ultra high definition (UHD) image is increasing in various applications. As the image data has high resolution and high quality, the amount of data increases relative to the existing image data. Therefore, when the image data is transmitted using a medium such as a wired / wireless broadband line or stored using an existing storage medium, The storage cost is increased. High-efficiency image compression techniques can be used to solve these problems caused by high-resolution and high-quality image data.

An inter picture prediction technique for predicting a pixel value included in a current picture from a previous or a subsequent picture of a current picture using an image compression technique, an intra picture prediction technique for predicting a pixel value included in a current picture using pixel information in the current picture, There are various techniques such as an entropy encoding technique in which a short code is assigned to a value having a high appearance frequency and a long code is assigned to a value having a low appearance frequency. Image data can be effectively compressed and transmitted or stored using such an image compression technique.

On the other hand, demand for high resolution images is increasing, and demand for stereoscopic image contents as a new image service is also increasing. Video compression techniques are being discussed to effectively provide high resolution and ultra-high resolution stereoscopic content.

In order to utilize the high-resolution and super-high-resolution stereoscopic image contents for broadcasting service or VOD (Video on Demand) service, fast coding process is required. For this fast encoding process, a parallel encoding technique of image data can be used.

However, in the case where the image is divided into predetermined units and the parallel encoding is performed on the divided areas, there is no information of the pixels that refer to the boundary or the edge of the divided area in the encoding process, The deterioration of the image quality occurs at the boundary portion or the edge portion of the region.

A technical object of the present invention is to provide a method and apparatus capable of overcoming image quality deterioration even when parallel encoding is performed on a divided area unit basis.

The technical objects to be achieved by the present disclosure are not limited to the above-mentioned technical subjects, and other technical subjects which are not mentioned are to be clearly understood from the following description to those skilled in the art It will be possible.

According to one aspect of the present disclosure, a parallel encoding method of an image can be provided. The method includes the steps of detecting characteristics of at least one picture input in advance and storing characteristics of the at least one picture in advance in a database; constructing a plurality of divided pictures by dividing an input picture; Detecting a characteristic of each of the divided pictures of the plurality of divided pictures; detecting, from the database, the previously stored picture corresponding to the detected characteristics of the plurality of divided pictures; A step of constructing a boundary prediction picture reflecting the pixel values of the detected pre-stored picture, and a process of performing parallel coding on the plurality of divided pictures using the boundary prediction picture as a reference picture.

The features briefly summarized above for this disclosure are only exemplary aspects of the detailed description of the disclosure which follow, and are not intended to limit the scope of the disclosure.

According to the present disclosure, it is possible to provide an image encoding method and apparatus capable of overcoming image deterioration even when parallel encoding is performed by dividing an image.

Furthermore, according to the present disclosure, a method and an apparatus capable of realizing high-speed and high-resolution high-speed encoding processing with minimum image quality loss can be provided.

The effects obtainable from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

1 is a block diagram showing a configuration of an image encoding apparatus according to an embodiment of the present disclosure.
2A to 2C are diagrams illustrating pictures processed in the image encoding apparatus according to an embodiment of the present disclosure.
3 is a flowchart illustrating a procedure of an image encoding method according to an embodiment of the present disclosure.
FIG. 4 is a diagram illustrating a computing environment to which an image encoding apparatus and method according to an embodiment of the present disclosure is applied.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, which will be easily understood by those skilled in the art. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear. Parts not related to the description of the present disclosure in the drawings are omitted, and like parts are denoted by similar reference numerals.

In the present disclosure, when an element is referred to as being "connected", "coupled", or "connected" to another element, it is understood that not only a direct connection relationship but also an indirect connection relationship May also be included. Also, when an element is referred to as " comprising "or" having "another element, it is meant to include not only excluding another element but also another element .

In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one element from another, and do not limit the order or importance of elements, etc. unless specifically stated otherwise. Thus, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly a second component in one embodiment may be referred to as a first component .

In the present disclosure, the components that are distinguished from each other are intended to clearly illustrate each feature and do not necessarily mean that components are separate. That is, a plurality of components may be integrated into one hardware or software unit, or a single component may be distributed into a plurality of hardware or software units. Thus, unless otherwise noted, such integrated or distributed embodiments are also included within the scope of this disclosure.

In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some may be optional components. Thus, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Also, embodiments that include other elements in addition to the elements described in the various embodiments are also included in the scope of the present disclosure.

The terms used in this disclosure can be defined as follows.

The input picture may be image data input for encoding.

The divided picture may be a picture formed by dividing the input picture into a plurality of areas.

The boundary area of the divided picture may be an area up to a predetermined number of pixels (or pixels) based on the outermost pixel (or pixel) of the divided picture.

The boundary prediction picture may be a picture including a border area between the divided picture and the divided picture.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a block diagram showing a configuration of an image encoding apparatus according to an embodiment of the present disclosure. 1, an image encoding apparatus 10 according to an embodiment of the present invention includes a picture division unit 11, an image analysis unit 13, a boundary region prediction unit 15, and a coding unit 17 .

The coding unit 17 can process parallel coding and the picture dividing unit 11 can divide the input picture into units of processing for performing parallel coding in the coding unit 17. [ More specifically, the picture dividing unit 11 can divide the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture division unit 11 divides a picture into a plurality of coding units, a prediction unit, and a combination of conversion units, and generates a coding unit, a prediction unit, and a conversion unit combination So that the picture can be encoded.

For example, one picture may be divided into a plurality of coding units. In order to divide a coding unit in a picture, a recursive tree structure such as a quad tree structure can be used. In a coding or decoding scheme in which one picture or a largest coding unit is used as a root and divided into other coding units A unit can be divided with as many child nodes as the number of divided coding units. Under certain constraints, an encoding unit that is no longer segmented becomes a leaf node. That is, when it is assumed that only one square division is possible for one coding unit, one coding unit can be divided into a maximum of four different coding units.

Hereinafter, in the embodiment of the present invention, a coding unit may be used as a unit for performing coding, or may be used as a unit for performing decoding.

The prediction unit may be one divided into at least one square or rectangular shape having the same size in one coding unit, and one of the prediction units in one coding unit may be divided into another prediction Or may have a shape and / or size different from the unit.

If a prediction unit performing intra prediction on the basis of an encoding unit is not the minimum encoding unit at the time of generation, intraprediction can be performed without dividing the prediction unit into a plurality of prediction units NxN.

The image analyzing unit 13 can detect characteristics of a scene or an object included in the image and store and manage characteristics of the scene or object included in the image from the input image. Since the image analyzing unit 13 can detect characteristics of scenes or objects through repeated learning of various images, characteristics of scenes or objects included in the images can be generalized. For example, the image analyzing unit 13 analyzes frequency information, a wave form, a pitch, a feature vector, a eigenface, a skin pattern, , Facial geometry, thermogram, edge information, and line information, for example.

In addition, the image analyzer 13 can detect characteristics of a scene or a part of an object included in the image, and can repeatedly learn a difference between the characteristic of the scene or the entire object included in the image. Accordingly, even if only characteristic parameters of a scene or a part of an object included in the image are detected, characteristics corresponding to the scene or the entire object corresponding thereto can be confirmed. For example, the image analysis unit 13 may perform various feature matching methods such as neural networks, a hidden markov model, deep learning, or maximum likelihood And the image corresponding to the scene or the object included in the input image can be detected from the database.

In consideration of this point, the image analyzing unit 13 receives a picture (hereinafter, referred to as a "divided picture") divided into a predetermined size by the picture dividing unit 11 and detects the characteristics of the input divided picture The scene or object corresponding to the characteristic of the divided picture can be detected. The image analysis unit 13 may provide the boundary region prediction unit 15 with an image including the detected scene or object.

The boundary area prediction unit 15 generates a boundary prediction picture reflecting the boundary area of the divided picture so that the image including the detected scene or object can be used as a reference picture for coding the divided picture , And provide the generated boundary predictive picture to the coding unit 17. [ In particular, the boundary area predicting unit 15 can check information (e.g., color difference information of pixels) of a neighboring area and a divided picture to be coded. At this time, information of the image including the detected scene or object (E.g., chrominance information of pixels) can be used to predict the boundary area of the divided picture.

The encoding unit 17 performs inter-prediction, intra-prediction, transform processing of a source block and a residual block (DCT (Discrete Cosine Transform), DST (Discrete Sine Transform) KLT (Karhunen-Loeve Transform, etc.)), quantization processing, and the like.

In particular, the coding unit 17 can process parallel coding in units of divided pictures and can process parallel coding in a data-level parallel coding method. The encoding unit 17 can perform predictive encoding such as inter prediction and intra prediction. In the process of performing the predictive encoding process for each divided picture, the boundary predictor 15 provides the boundary predictive picture Predictive encoding processing can be performed.

(Hereinafter, referred to as an " input image ") on the basis of the data-level parallelization method and divides the divided data and the divided picture into different parallel processors (e.g., core and thread) So that the divided images can be encoded in parallel. In this manner, when encoding the image based on the data-level parallelization method, there is no reference pixel in the encoding process for the pixel area located near the boundary surface or the boundary surface of the divided picture, And uses it for encoding processing. When the virtual reference sample is used as described above, the information of the virtual reference sample (e.g., the color difference information of the pixel) is not the same as the information included in the actual image, and image deterioration occurs.

However, according to the embodiment of the present disclosure, since the boundary predictive picture provided by the boundary area prediction unit 15 is detected based on the characteristics of a scene or an object included in various images, Similarity with the actual information (e.g., the chrominance information of the pixel) of the region where the pixel is located may be high. Therefore, the image encoding apparatus according to an embodiment of the present disclosure can solve the problem of image quality deterioration that occurs in the encoding process.

In the embodiment of the present disclosure, in consideration of encoding the moving picture data, the image coding apparatus may be configured such that the coding unit 17 performs inter prediction, intra prediction, conversion processing of the original block and the residual block (DCT (Discrete Cosine Transform) DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform, etc.)), quantization processing, and the like. However, the present disclosure is not limited thereto. The image encoding apparatus according to an embodiment of the present disclosure may be applied to parallel encoding of still image data as well as moving image data. In this case, the encoding unit 17 may process an operation of parallelly encoding still image data.

Hereinafter, the operation of the image encoding apparatus according to one embodiment of the present disclosure described above will be described with reference to FIGS. 2A to 2C.

2A to 2C are diagrams illustrating pictures processed in an image encoding apparatus according to an embodiment of the present disclosure. FIG. 2A illustrates an input picture 210, FIG. 2B illustrates a divided picture, Illustrates a boundary predictive picture in which a boundary region is predicted.

First, when the input picture 210 is inputted to the picture coding apparatus, the picture dividing unit 11 divides the input picture 210 into a predetermined size in consideration of a processing unit for performing parallel coding, have.

The picture dividing unit 11 divides the input picture 210 into four divided pictures 221, 222, and 223 (for example, , 224).

The split pictures 221, 222, 223 and 224 constituted by the picture dividing section 11 are provided to the image analyzing section 13 and the encoding section 17 as described above.

In response to this, the image analyzing unit 13 performs image analysis on each of the divided pictures, that is, the first through fourth divided pictures 221, 222, 223, and 224, Can be detected. For example, the frequency information, the wave form, the pitch, the feature vector, and the eigenface of the first divided picture 221 (or the scene or object included in the first divided picture 221) facial geometry, thermogram, edge information, line information, and the like can be extracted from the image data. Then, the image analysis unit 13 detects an image having characteristics corresponding to the characteristics of the extracted first divided picture 221 (or the scene or object included in the first divided picture 221) from the database . The image analysis unit 13 may provide the boundary region prediction unit 15 with an image corresponding to the first divided picture 221 (or the scene or object included in the first divided picture 221).

The boundary area predicting section 15 predicts from the picture corresponding to the scene or object included in the first divided picture 221 the pixel corresponding to the boundary area used (referred to) in the coding process of the first divided picture 221 It is possible to construct the first boundary predictive picture 231 in which the color difference information of the first boundary picture 221 is detected and reflected in the first divided picture 221. [ The first boundary predictive picture 231 constructed as described above can be provided to the encoding unit 17. The encoding unit 17 encodes the first divided picture 221 and outputs the first boundary predictive picture 231 ) Can be used.

In this way, the image analyzing unit 13 can form the second to fourth boundary prediction pictures 232, 233, and 234 corresponding to the second to fourth divided pictures 222, 223, and 224, respectively , And the second to fourth boundary prediction pictures 232, 233, and 234 can be used as reference pictures in the encoding process of the encoding unit 17.

Hereinafter, a video encoding method according to an embodiment of the present disclosure will be described.

The image encoding method according to an embodiment of the present disclosure can be performed by the above-described image encoding apparatus.

3 is a flowchart illustrating a procedure of an image encoding method according to an embodiment of the present disclosure.

Referring to FIG. 3, in step S301, the image encoding apparatus may receive an image or a picture to be encoded.

In step S302, the image encoding apparatus can form the divided pictures 221, 222, 223, and 224 by dividing the input picture 210 into a predetermined size in consideration of a processing unit for performing parallel encoding.

At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture division unit 11 divides a picture into a plurality of coding units, a prediction unit, and a combination of conversion units, and generates a coding unit, a prediction unit, and a conversion unit combination So that the picture can be encoded.

For example, one picture may be divided into a plurality of coding units. In order to divide a coding unit in a picture, a recursive tree structure such as a quad tree structure can be used. In a coding or decoding scheme in which one picture or a largest coding unit is used as a root and divided into other coding units A unit can be divided with as many child nodes as the number of divided coding units. Under certain constraints, an encoding unit that is no longer segmented becomes a leaf node. That is, when it is assumed that only one square division is possible for one coding unit, one coding unit can be divided into a maximum of four different coding units.

Hereinafter, in the embodiment of the present invention, a coding unit may be used as a unit for performing coding, or may be used as a unit for performing decoding.

The prediction unit may be one divided into at least one square or rectangular shape having the same size in one coding unit, and one of the prediction units in one coding unit may be divided into another prediction Or may have a shape and / or size different from the unit.

In step S303, the image encoding apparatus can perform the image analysis on each of the divided pictures 221, 222, 223, and 224 to extract characteristics for the divided pictures 221, 222, 223, and 224, respectively. For example, the image encoding apparatus may generate frequency information, a wave form, a pitch, and a feature vector (for example, a pitch, a pitch, and a pitch) for each of the divided pictures 221, 222, 223, and 224 feature vector, eigenface, skin pattern, facial geometry, thermogram, edge information, and line information can be extracted.

In step S304, the image encoding apparatus refers to the database to generate a video having characteristics corresponding to the characteristics of the divided pictures 221, 222, 223, and 224 (or scenes or objects included in the divided picture) extracted in step S303 (Or picture) can be detected.

Next, in step S305, the image encoding apparatus generates an image (or a picture) corresponding to an image (or picture) having characteristics corresponding to the characteristics of the divided pictures 221, 222, 223, and 224 The chrominance information of the pixel corresponding to the boundary area used (referred to) in the coding process of the divided pictures 221, 222, 223 and 224 is detected from the image and is reflected on the divided pictures 221, 222, 223 and 224 The boundary prediction picture 230 can be constructed.

Next, in step S306, the image encoding apparatus performs inter-prediction, intra-prediction, transform processing (DCT (Discrete Cosine Transform), and DST (Discrete Cosine Transform) on the divided blocks 221, 222, 223, Sine Transform), KLT (Karhunen-Loeve Transform, etc.), quantization processing, and the like.

Furthermore, the image encoding apparatus can process parallel pictures by the data-level parallel encoding method on the divided pictures 221, 222, 223, and 224. Particularly, the image encoding apparatus can perform predictive encoding processing such as inter prediction and intra prediction. In the process of performing the predictive encoding processing of the divided pictures, the predictive encoding processing can be performed using the boundary predictive picture 230 have.

It is possible to parallelly encode the divided pictures by dividing the input picture into a plurality of units based on the data level parallelization method and then allocating the divided data, that is, the divided pictures to different parallel processors (e.g., core and thread) have. In this manner, when encoding the image based on the data-level parallelization method, there is no reference pixel in the encoding process for the pixel area located near the boundary surface or the boundary surface of the divided picture, And uses it for encoding processing. When the virtual reference sample is used as described above, the information of the virtual reference sample (e.g., the color difference information of the pixel) is not the same as the information included in the actual image, and image deterioration occurs.

However, according to an embodiment of the present disclosure, since the boundary predictive picture 230 constituted by the image encoding apparatus is detected based on characteristics of a scene or an object included in various images, the divided pictures 221, 222, 223, 224) or the actual information (for example, the chrominance information of the pixel) of the region located near the interface. Therefore, according to the image encoding method according to an embodiment of the present disclosure, image quality deterioration occurring in the encoding process of the input picture can be prevented when the input picture is parallel-encoded.

FIG. 4 is a diagram illustrating a computing environment to which an image encoding apparatus and method according to an embodiment of the present disclosure is applied.

The computing environment illustrated in FIG. 4 is an example of an operating environment in which an image encoding apparatus and method may be implemented and does not limit the scope of use or functionality of the present disclosure.

Examples of computing systems, environments, and / or configurations that may be suitably employed in an image encoding apparatus and method in accordance with an embodiment of the present disclosure include, but are not limited to, a personal computer, a server computer, a handheld or laptop device, Based systems, programmable consumer electronics, network personal computers, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

An apparatus and method for encoding an image, according to one embodiment of the present disclosure, will be described generally in connection with computer-executable instructions, such as program modules, being executed by one or more computers or other devices. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functions of the program modules may be combined or distributed according to various circumstances.

Referring to FIG. 4, computing device 400 typically includes at least one processing unit 402 and memory 404.

Depending on the configuration and type of the computing device, the memory may be a volatile memory such as a RAM, a nonvolatile memory such as a ROM, a flash memory, or the like, or a combination thereof. In addition, the computing device 400 may include additional (e.g., removable and / or stationary) storage devices such as magnetic or optical disks or tapes. The additional storage device may include a removable storage device 408 and a stationary storage device 410.

The memory 904, the removable storage device 408 and the fixed storage device 410 may all be provided as computer storage media and may be embodied in a variety of forms, including RAM, ROM, EEPROM, flash memory or other memory technology, CD- versatile disk, or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other medium that can be accessed by device 400 and store the desired information .

The computing device 400 may include a communication unit 412 that enables communication with other devices, such as other computing devices, via a wired or wireless network. The communication unit 412 typically implements computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, Information transmission medium. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed to encode information in the signal. By way of example, the communication unit 412 may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency, infrared, or other wireless media.

The computing device 400 may include an input device 114 such as a keyboard, a mouse, a pen, a voice input device, a touch input device, a laser range finder, an infrared camera, In addition, the computing device 400 may include an output device 416 such as a display, a speaker, or a printer.

At least one processing unit 402 included in the computing device 400 may process operations corresponding to the image encoding apparatus and method according to one embodiment of the present disclosure. In particular, the processing unit 402 of the computing device 400 according to one embodiment of the present disclosure detects the characteristics of at least one picture previously entered and pre-stores the characteristics for the at least one picture in the database An operation of dividing an input picture to form a plurality of divided pictures, an operation of detecting a characteristic of each of the plurality of divided pictures, and an operation of extracting, from the database, An operation of detecting a stored picture and an operation of constructing a boundary predictive picture reflecting a pixel value of the detected pre-stored picture in a boundary area of the plurality of divided pictures; and using the boundary predictive picture as a reference picture, It is possible to process an operation of processing parallel coding for a plurality of divided pictures.

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to illustrate representative aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

Claims (1)

A method of encoding an image,
Detecting characteristics of at least one picture input in advance and storing characteristics of the at least one picture in a database in advance;
Dividing an input picture into a plurality of divided pictures;
Detecting characteristics of each of the plurality of divided pictures;
Detecting, from the database, the previously stored picture corresponding to the detected characteristics of the plurality of divided pictures;
Constructing a boundary predictive picture in which a pixel value of the detected pre-stored picture is reflected in a boundary area of the plurality of divided pictures;
And processing the parallel coding for the plurality of divided pictures using the boundary predictive picture as a reference picture.

Images