KR20190063451A - Method and apparatus for measuring image quality base on perceptual sensitivity - Google Patents

Abstract

A method and an apparatus for measuring the image quality of an image considering perceptual sensitivity are disclosed. The apparatus comprises: a spatial distortion analyzing unit for generating spatial distortion information based on spatial feature information of an original image, spatial feature information of a comparative image, and spatial perceptual sensitivity-based information of the original image; a temporal distortion analyzing unit for generating spatial distortion information based on temporal feature information of the original image, temporal feature information of the comparative image, and temporal perceptual sensitivity-based information of the original image; and an index calculating unit for generating an image quality index for the subjective image quality based on the temporal distortion information and the spatial distortion information.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and apparatus for measuring image quality,

The following embodiments relate to a method and apparatus for image processing, and more particularly, to a method and apparatus for measuring the image quality of an image that considers cognitive sensitivity.

With the continuous development of the information and telecommunication industry, broadcasting service with HD (High Definition) resolution spread worldwide. With this proliferation, many users become accustomed to high resolution and high quality images and / or video.

In order to meet the users' demand for high image quality, many organizations are spurring development on next generation image devices. In addition to High Definition TV (HDTV) and Full HD (FHD) TVs, users' interest in ultra high definition (UHD) TVs with more than four times the resolution of FHD TVs As the interest increases, there is a need for image encoding / decoding techniques for images with higher resolution and image quality. Also, in video service, "image quality" of video becomes a very important competitive power.

Research areas related to image quality include the field of image compression, which is required to provide optimal image quality under limited capacities, and the field of super-resolution image processing, which enhances image quality.

In the field of image compression and image processing, an index indicating the image quality is required for verifying the performance of the developed technology or for selecting a mode in the developed technology. Also, as these indicators reflect the subjective image quality performance, the developed technology can provide superior subjective image quality performance.

Full reference image quality indicators such as the existing mean squared error (MSE), peak signal to noise ratio (PSNR), and structural similarity (SSIM) The method of measuring the difference between the target images has advantages of being intuitive, but there is a large difference between the measured values and the cognitive quality according to the index, and relatively low accuracy in comparison between images having different characteristics .

One embodiment of the present invention can provide an apparatus and method for predicting image quality of an image by extracting various temporal feature information and spatial feature information from an image and applying a cognitive sensitivity based weighting suitable for the extracted feature information.

One embodiment can provide an apparatus and method for determining a sensitive region from a cognitive point of view in an image and generating an index that takes temporal distortion and spatial distortion in the identified region into consideration.

One embodiment can provide an apparatus and method for calculating an index of reliability even in comparison between images of different characteristics having similar tendencies to the actual perceived image quality of the image.

A spatial distortion analyzer for generating spatial distortion information based on spatial feature information of an original image, spatial feature information of a comparison image, and spatial perception sensitivity information of the original image; A temporal distortion analyzer for generating spatial distortion information based on temporal feature information of the original image, temporal feature information of the comparative image, and temporal cognitive sensitivity based information of the original image; And an index calculator for generating an index of quality of subjective image quality based on the temporal distortion information and the spatial distortion information.

In addition, there is further provided another method, apparatus, system for implementing the invention and a computer readable recording medium for recording a computer program for executing the method.

There is provided an apparatus and method for predicting image quality of an image by extracting various temporal feature information and spatial feature information from an image and applying a weight based on cognitive sensitivity to the extracted feature information.

There is provided an apparatus and method for discriminating a sensitive region from a cognitive viewpoint in an image and generating an indicator that takes temporal distortion and spatial distortion in the discriminated region into consideration.

There is provided an apparatus and a method for calculating an index of reliability even in a comparison between images of different characteristics having a similar tendency to the actual cognitive image quality of the image.

1 shows a structure of an electronic device according to an embodiment.
2 shows a structure of a processing unit according to an example.
3 is a flow diagram of a method for deriving an indicator for image quality according to an embodiment.
Figure 4 shows that the dominant angle of the pixel according to one example does not determine the pixel as a strong HV pixel.
FIG. 5 shows that the dominant angle of the pixel according to an example determines that the pixel is determined as a strong HV pixel.
FIG. 6 shows a comparison between the degree of distortion for the HV plane and the actual perceived image quality according to an example.
Figure 7 shows a method for determining the degree of distortion due to block artifacts according to a cognitive view according to an example.
8 shows a spatial and temporal frame of an image according to an example.
Figure 9 shows a row plane of temporal frames according to an example.
FIG. 10 shows applying a cognitive sensitivity-based weighting to the degree of distortion for a column plane of temporal frames according to an example.

In the following, embodiments will be described in detail with reference to the accompanying drawings. It should be understood that the embodiments are different, but need not be mutually exclusive.

The terms used in the embodiments can be interpreted based on the actual meaning of the terms that are not the names of simple terms and the contents throughout the specification.

In embodiments, the connection relationship for a particular portion and the other portion may include an indirect connection relationship that is connected via another portion therebetween, in addition to the direct connection relationship between the specific portion and the other portion. Like reference numerals in the drawings denote like elements.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained.

In the drawings, like reference numerals refer to the same or similar functions throughout the several views. The shape and size of the elements in the figures may be exaggerated for clarity.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

The following detailed description of exemplary embodiments refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the location or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the embodiments. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the exemplary embodiments is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained.

In the drawings, like reference numerals refer to the same or similar functions throughout the several views. The shape and size of the elements in the figures may be exaggerated for clarity.

The terms used in the examples are intended to illustrate the embodiments and are not intended to limit the invention. In the examples, the singular includes the plural unless otherwise stated in the specification. It is noted that the terms "comprises" and / or "comprising" used in the specification are intended to be inclusive in a manner similar to the components, steps, operations, and / And that additional configurations may be encompassed within the scope of the embodiments of the exemplary embodiments or the technical ideas of the exemplary embodiments. When it is mentioned that a component is "connected" or "connected" to another component, the two components may be directly connected or connected to each other, It is to be understood that other components may be present in the middle of the components.

The terms first and second, etc. may be used to describe various components, but the components should not be limited by the terms above. The above terms are used to distinguish one component from another. For example, without departing from the scope of the right, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

In addition, the elements shown in the embodiments are shown independently to represent different characteristic functions, and do not mean that each element is composed of separate hardware or a single software constituent unit. That is, each component is listed as each component for convenience of explanation. For example, at least two of the components may be combined into a single component. Also, one component can be divided into a plurality of components. The integrated embodiments and the separate embodiments of each of these components are also included in the scope of the right without departing from the essence.

Also, some components are not essential components to perform essential functions, but may be optional components only to improve performance. Embodiments may be implemented only with components that are essential to implementing the essentials of the embodiments, and structures within which the optional components are excluded, such as, for example, components used only for performance enhancement, are also included in the scope of the right.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to facilitate embodiments of the present invention by those skilled in the art. In the following description of the embodiments, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear.

The embodiment relates to an index for measuring the cognitive image quality of a video. It extracts various temporal feature information and spatial feature information from an image and applies a cognitive sensitivity-based weight to the extracted feature information to predict the image quality have.

Since the proposed index identifies a sensitive region from a cognitive viewpoint in the image and considers temporal distortion and spatial distortion in the discriminated region, the index calculated for the image is similar to the actual cognitive image quality of the image Lt; / RTI > Therefore, the proposed indicator can also be reliable in comparison between images of different characteristics.

1 shows a structure of an electronic device according to an embodiment.

The electronic device 100 may include at least a part of the processing unit 110, the communication unit 120, and the storage unit 130 as components. The components may communicate with each other via one or more communication buses or signal lines.

The electronic device 100 may be an image processing device that derives an index of image quality.

The components shown relative to the electronic device 100 in FIG. 1 may be but one example. Not all of the components shown may be necessary for the electronic device 100. Electronic device 100 may have more or fewer components than those shown in FIG. Further, two or more components shown in Fig. 1 may be combined. In addition, the components may be configured or arranged differently than shown in FIG. Each component may be implemented in hardware, including one or more signal processing and / or application specific integrated circuits (ASICs), or in software, or in a combination of hardware and software.

The processing unit 110 may process a job required for the operation of the electronic device 100. [ The processing unit 110 may execute the code of the operation or step of the processing unit 110 described in the embodiments.

The processing unit 110 may be a processor that is capable of inputting to, outputting from, or generating and processing signals, data, or information generated by the electronic device 100, Comparison, judgment, and the like related to the inspection. In other words, in the embodiment, the generation and processing of data or information and the inspection, comparison and judgment related to data or information can be performed by the processing unit 110. [

For example, the processing unit 110 may be at least one processor.

The processor may be a hardware processor and may be a central processing unit (CPU). The processor may be plural. Alternatively, the processor may include a plurality of cores and may provide multi-tasking that simultaneously executes a plurality of processes and / or a plurality of threads. At least some of the steps of the embodiments may be performed in parallel for a plurality of objects through a plurality of processors, a plurality of cores, a plurality of processes, and / or a plurality of threads.

For example, the processing unit 110 may execute the code of the operation or step of the electronic device 100 described in the embodiments.

For example, the processing unit 110 may run a program. The processing unit 110 can execute a code (code) constituting a program. The program may include an operating system (OS), a system program, an application, and an app of the electronic device 100.

In addition, the processing unit 110 may control other components of the electronic device 100 for the function of the processing unit 110 described above.

The communication unit 120 may receive data or information used for operation of the electronic device 100 and may transmit data or information used for the operation of the electronic device 100. [

The communication unit 120 may transmit data or information to another device in the network to which the electronic device 100 is connected, and may receive data or information from another device. In other words, in the embodiment, the transmission or reception of data or information can be performed by the communication unit 120. [

For example, the communication unit 120 may be a networking chip, a networking interface, or a communication port.

The storage unit 130 may store data or information used for operation of the electronic device 100. In an embodiment, the data or information that electronic device 100 has may be stored in storage 130.

For example, the storage unit 130 may be a memory. The storage unit 130 may include a built-in storage medium such as a RAM and a flash memory, and may include a removable storage medium such as a memory card or the like.

The storage unit 130 may store at least one program. The processing unit 110 may execute at least one program. The processing unit 110 can read the code of at least one program from the storage unit 130 and execute the readout code.

Operations, functions, and characteristics of the processing unit 110, the communication unit 120, and the storage unit 130 of the electronic device 100 will be described in detail below with reference to embodiments.

2 shows a structure of a processing unit according to an example.

The processing unit 110 includes an image characteristic information extracting unit 210, a spatial cognitive sensitivity calculating unit 220, a temporal cognitive sensitivity calculating unit 230, a spatial distortion analyzing unit 240, a temporal distortion analyzing unit 250, (260).

Alternatively, the image feature information extracting unit 210, the spatial cognitive sensitivity calculating unit 220, the temporal cognitive sensitivity calculating unit 230, the spatial distortion analyzing unit 240, the temporal distortion analyzing unit 250, and the indicator calculating unit 260 May be at least one program executed by the processing unit 210, a part of at least one program, or at least one of the modules.

The original image may be input to the image feature information extracting unit 210, the spatial cognitive sensitivity calculating unit 220, and the temporal cognitive sensitivity calculating unit 230.

A comparison image may be input to the image feature information extracting unit 210.

The spatial feature information of the original image and the spatial feature information of the comparison image output from the image feature information extracting unit 210 may be input to the spatial distortion analyzer 240.

The temporal feature information of the original image and the temporal feature information of the compared image output from the image feature information extracting unit 210 may be input to the temporal distortion analyzer 250.

The image quality index can be output from the index calculation unit 260. [

A spatial cognitive sensitivity calculation unit 220, a temporal cognitive sensitivity calculation unit 230, a spatial distortion analysis unit 240, a temporal distortion analysis unit 250, and an index calculation unit 260. The image characteristic information extraction unit 210, the spatial cognitive sensitivity calculation unit 220, Specific operations and functions will be described in more detail below.

3 is a flow diagram of a method for deriving an indicator for image quality according to an embodiment.

The communication unit 120 can receive the original image and the comparison image.

The comparison image may be an encoded image. Alternatively, the comparison image may be an image transmitted through a network or an image subjected to specified image processing. Alternatively, the comparison image may be a damaged image.

In operation 310, the image feature information extraction unit 210 may generate spatial feature information of an original image, spatial feature information of a comparative image, temporal feature information of an original image, and temporal feature information of a comparative image.

The spatial feature information of the original image may include the SI plane and the HV plane of the original image.

The spatial feature information of the comparison image may include the SI plane and the HV plane of the comparison image.

The image feature information extraction unit 210 may generate spatial feature information of an image using a spatial information filter defined as Equation 1 below.

[Equation 1]

h (x) may be the filter coefficient value for the xth pixel of the image. If the size of the filter is L, the value of x can range from -L / 2 to L / 2.

Alternatively, the image feature information extraction unit 210 may generate spatial feature information of the image using various edge extraction filters that can recognize the structure information of the image.

The planes obtained by applying these filters in the horizontal and vertical directions of the image can be referred to as I _H and I _V , respectively.

Also, the SI plane can be defined as shown in Equation 2 below.

[Equation 2]

The SI plane may contain information about the dominant structure in the image. Therefore, the SI plane may be spatial feature information that can be used for predicting the perceived quality of an image.

The image feature information extracting unit 210 can obtain a horizontal-vertical (HV) plane that identifies strong edges of the vertical component and the horizontal component in the image along with the SI plane. The image feature information extracting unit 210 can utilize the HV plane to predict the degree of distortion due to block artifacts generated when compressing an image.

The construction of the HV plane will be described in detail below with reference to FIGS. 4 and 5. FIG.

The temporal feature information of the original image may include temporal frame information of the original image. The temporal feature information of the target image may include temporal frame information of the target image.

In step 320, the spatial perception sensitivity calculation unit 220 may generate spatial perception sensitivity based information on the original image using the original image.

The spatial cognitive sensitivity based information for the original image may include a spatial cognitive sensitivity based weight map for the original image.

In step 330, the temporal perception sensitivity calculation unit 230 may generate temporal perceptual sensitivity-based information on the original image using the original image.

The temporal cognitive sensitivity based information may include a temporal cognitive sensitivity based weight map for the original image.

The temporal perception sensitivity calculation unit 230 can determine an area having a large motion at the center of the original image considering the perception characteristics of the original image. The temporal perception sensitivity calculation unit 230 can generate information on a region where the discriminated motion is large.

The temporal cognitive sensitivity-based information may include information about a region with large motion.

The temporal perception sensitivity calculation unit 230 may generate a residual (or difference) image between adjacent frames of the original image in order to discriminate an area having a large motion at the center of the original image.

In order to give a weight to the center of the original image, the temporal perception sensitivity calculation unit 230 may multiply the residual image between adjacent frames by a weighting function of the Gaussian distribution as expressed by Equation (3).

[Equation 3]

N may be the total number of rows or the total number of columns. n may be a current row or column.

The temporal perception sensitivity calculation unit 230 can use the residual image to discriminate an area having a large motion at the center of the original image.

In step 340, the spatial distortion analyzer 240 may receive the spatial feature information of the original image and the spatial feature information of the comparison image from the image feature information extractor 210. [

The spatial feature information of the original image may include the SI plane and the HV plane of the original image.

The spatial feature information of the comparison image may include the SI plane and the HV plane of the comparison image.

In addition, the spatial distortion analyzer 240 may receive cognitive sensitivity-based information on the original image from the spatial cognitive sensitivity calculation unit 220. [

The cognitive sensitivity based information for the original image may include a spatial cognitive sensitivity based weight map for the original image.

The spatial distortion analyzer 240 may generate the spatial distortion information based on the spatial feature information of the original image, the spatial feature information of the comparative image, and the cognitive sensitivity based information on the original image.

The spatial distortion analyzer 240 can analyze the degree of the spatial distortion based on the spatial feature information of the original image, the spatial feature information of the comparative image, and the cognitive sensitivity information of the original image, Lt; / RTI >

The spatial distortion information may include the results of an analysis of the degree of spatial distortion and may include numerical values related to the degree of spatial distortion.

The spatial distortion analyzer 240 may compare the SI plane of the original image and the SI plane of the comparison image to derive the degree of difference between the SI plane of the original image and the SI plane of the comparison image.

The degree of difference between the SI plane of the original image and the SI plane of the comparison image may indicate the degree of reduction or improvement of the overall sharpness in the images.

The spatial distortion analyzer 240 may perform a comparison between the HV plane of the original image and the HV plane of the comparison image to derive the degree of difference between the HV plane of the original image and the HV plane of the comparison image.

The degree of the difference between the HV plane of the original image and the HV plane of the comparison image may indicate the degree of occurrence of block artifacts in the images.

In comparing the planes of the images, the degree of difference between the images may be the difference value between the images. That is to say, comparing the planes of the images to derive the degree of difference between the images may be to calculate the difference value between the images.

Also, the degree of difference between images can be derived by using a ratio comparison function as shown in Equation (4) below.

[Equation 4]

Also, the degree of difference between images can be derived using a log comparison function as shown in Equation (5) below.

[Equation 5]

f _o can be the SI plane value or the HV plane value of the original image.

f _p may be the SI plane value of the comparison image or the HV plane value.

The ratio comparison function and the log comparison function can reflect the visual masking effect of the image at a certain level. Such visual masking may mean that the degree of difference in the specified region appearing in the SI plane or the HV plane is cognitively less exposed when the complexity of the specified region in the original image is large.

A comparison between the degree of distortion with respect to the HV plane and the actual perceived quality is described in more detail below with reference to Figures 6 and 7.

The spatial distortion information may include HV plane distortion information and SI plane distortion information. The spatial distortion analyzer 240 may transmit the HV plane distortion information and the SI plane distortion information to the index calculator 260.

The HV plane distortion information may be information indicating the degree of distortion with respect to the HV plane. The HV plane distortion information may be one in which the degree of distortion of the HV plane is converted into a single representative number through a weighted average or a percentile. Alternatively, the degree of distortion with respect to the HV plane can be used and transmitted as HV plane distortion information in a planar form.

The SI plane distortion information may be information indicating the degree of distortion with respect to the SI plane. The SI plane distortion information may be such that the degree of distortion of the SI plane is converted into a single representative number through weighted average or percentile. Alternatively, the degree of distortion with respect to the SI plane can be used and transmitted as the plane distortion information as the plane distortion information.

In step 350, the temporal distortion analyzer 250 may receive the temporal feature information of the original image and the temporal feature information of the comparative image from the image feature information extractor 210.

The temporal feature information of the original image may include temporal frames of the original image. The temporal feature information of the comparison image may include temporal frames of the comparison image.

The temporal distortion analyzer 250 may receive the temporal cognitive sensitivity-based information for the original image from the temporal cognitive sensitivity calculator 230.

The temporal cognitive sensitivity based information for the original image may include a temporal cognitive sensitivity based weight map for the original image.

The temporal distortion analyzer 250 may generate the temporal distortion information based on the temporal frames of the original image, the temporal frames of the compared image, and the temporal perceptual sensitivity-based weight map of the original image.

The temporal distortion analyzer 250 can analyze the temporal distortion based on the temporal frames of the original image, the temporal frames of the compared image, and the temporal perceptual sensitivity-based weight map of the original image, And outputs the related values to the index calculation unit 260. [

The temporal distortion information may include the results of an analysis of the degree of temporal distortion and may include values related to the degree of temporal distortion.

In comparing the temporal frames of the original image and the temporal frames of the comparison image, the temporal distortion analyzer 250 may calculate a difference value between temporal frames of the original image and temporal frames of the compared image, 250), it is possible to compare the temporal frames of the original image and the temporal frames of the comparison image using various other indicators including comparison functions.

The temporal distortion analyzer 250 may receive information on a region in which the motion of the center of the original image is large from the temporal perception sensitivity calculator 230. [

The temporal distortion information may include information on a region having a large motion.

The temporal distortion analyzer 250 can calculate the degree of distortion of the temporal frames with respect to the row plane.

In addition, the temporal distortion analyzer 250 can calculate the degree of distortion of the temporal frames with respect to the column plane.

The row and column planes of the frames are described in more detail below with reference to Figures 8 and 9.

The temporal distortion analyzing unit 250 analyzes the degree of distortion of the center of the original image using the information of the large region and the degree of distortion of the center of the original image, Can be given a large weight.

The assignment of weights will be described in more detail below with reference to FIG.

By the weighting, the degree of distortion of the temporal specific information from the cognitive viewpoint can be more accurately measured.

The temporal distortion information may include row plane distortion information and thermal plane distortion information of the computed temporal frames.

The temporal distortion analyzer 250 may transmit the row plane distortion information and the column plane distortion information of the calculated temporal frames to the index calculator 160. [

The row plane distortion information may be information indicating the degree of distortion of the row plane. The degree of distortion of the row plane may be a degree of distortion of the row plane converted into a single representative number through a weighted average or a percentile. Alternatively, the degree of distortion with respect to the row plane can be used and conveyed as row plane distortion information in a planar form.

The thermal plane distortion information may be information indicating the degree of distortion with respect to the thermal plane. The degree of thermal plane distortion may be a degree of distortion of the thermal plane converted into a single representative number through a weighted average or a percentile. Alternatively, the degree of distortion with respect to the thermal plane can be used and transmitted as the thermal plane distortion information in a planar form.

In step 360, the index calculator 260 may generate an image quality index for the subjective image quality based on the temporal distortion information and the spatial distortion information.

The image quality index may be a single number that predicts the subjective image quality.

The index calculator 260 may calculate a final image quality index by applying a weight based on an existing cognitive vision model to the temporal distortion information and / or the spatial distortion information.

Alternatively, the index calculator 260 can learn how to combine temporal distortion information and spatial distortion information by applying machine learning based on actual subjective image quality evaluation data.

For example, the spatial distortion information transmitted by the spatial distortion analyzer 240 and the temporal distortion information transmitted by the temporal distortion analyzer 250 may represent the degree of distortion of the feature information in the form of a numerical vector The indicator calculator 260 can be learned to output the image quality index using a neural network in the form of a fully connected (FC) layer.

For example, when the spatial distortion information transmitted by the spatial distortion analyzer 240 and the temporal distortion information transmitted by the temporal distortion analyzer 250 have a degree of distortion for the feature information in the entire plane, The unit 260 can be learned to output an image quality indicator using a neural network in the form of a Convolutional Neural Network (CNN).

Figure 4 shows that the dominant angle of the pixel according to one example does not determine the pixel as a strong HV pixel.

FIG. 5 shows that the dominant angle of the pixel according to an example determines that the pixel is determined as a strong HV pixel.

In Figs. 4 and 5, it has been shown whether the pixel is judged as a strong HV pixel according to the dominant angle of the pixel.

In order to calculate the HV plane, the image characteristic information extracting unit 210 may derive the size of the vertical direction component and the size of the horizontal direction component of each pixel of the image pixels.

The image feature information extracting unit 210 may obtain a dominant angle of a pixel through an arctangent operation using the size of the vertical direction component and the size of the horizontal direction component of the pixel.

The image feature information extracting unit 210 can construct the HV plane by determining whether the obtained dominant angle? Is close to the vertical direction or the horizontal direction.

The image feature information extracting unit 210 can determine whether the pixel is a strong HV pixel by determining whether the obtained dominant angle? Is close to the vertical direction or the horizontal direction.

The image feature information extracting unit 210 can classify the pixel into a strong HV pixel if the predominant angle [theta] of the pixel is close to the vertical direction or the horizontal direction. For example, the image feature information extracting unit 210 may classify the pixels into strong HV pixels if the dominant angle &thetas; of the pixels is smaller than the threshold &thetas; _threshold . For example, the image feature information extracting unit 210 may classify a pixel as not a strong HV pixel if the predominant angle [theta] of the pixel is equal to or greater than the threshold [theta] _threshold .

In order to obtain the dominant angle of the pixel and judge whether the pixel is a strong HV pixel, the following equation (6) may be used instead of the arc tangent operation having a high computational complexity.

[Equation 6]

According to Equation 6, the larger of I _H and I _V may be denominator, and the smaller value may be a molecule. That is to say, irrespective of the vertical direction and the horizontal direction, the value on which the judgment of the pixel is based can be a value in which a larger value of I _H and I _V is divided and a smaller value is a numerator.

It is possible to judge whether the horizontal component or the vertical component of the pixel is strong by comparing the value of the judgment of the pixel with the threshold of tan (? _Threshold ).

FIG. 6 shows a comparison between the degree of distortion for the HV plane and the actual perceived image quality according to an example.

For an SI plane that computes the degree of improvement or reduction in overall sharpness in an image, the values computed using the comparison functions described above with reference to step 340 of FIG. 8 may show a similar tendency to perceptual quality .

On the other hand, according to at least certain examples, in the case of the HV plane for measuring block artifact distortion, the values calculated using the above-mentioned comparison functions may differ from the actual cognitive distortion.

Fig. 6 is an example of a comparison between the degree of distortion for the HV plane and the actual cognitive quality.

As shown in FIG. 6, it can be seen from the numerical viewpoint that the distortion caused by the block artifacts is high in the area indicated by the yellow circles. From a cognitive point of view, however, distortions may appear more noticeably in edge and texture areas in a smooth background represented by a green circle than in a flat area represented by a yellow circle.

That is to say, in a flat area represented by a yellow circle, even if the degree of distortion is high, the perceived image quality may not be significantly deteriorated. In the edge and texture areas indicated by the green circles, even if the degree of distortion is low, the perceived image quality may be greatly deteriorated.

As part of the solution to these examples, regions where the coding distortion is noticeable can be determined.

Referring to the above-described step 320, spatial cognitive sensitivity based information may include information of visible regions where coding distortion is noticeable.

The spatial cognition sensitivity calculation unit 220 can determine an area in which the coding distortion such as block artifacts is noticeable according to cognitive viewpoints in the original image and transmits the information of the discriminated region to the spatial distortion analysis unit 240 .

When considered from a spatial point of view, the region where the distortion is noticeable may be a flat or regular region where the edge or texture is not random. In such a region, if a structure such as an edge or a texture of an image is deformed due to coding distortion or the like, the structure can be easily visualized because it deviates from a pattern expected from a cognitive point of view.

In this way, it is possible to identify regions that are not spatially random within the image, discriminate edge or texture regions existing in the region, and provide a high weight, thereby further considering information loss for a cognitively important region.

The spatial randomness can be calculated for a pixel as shown in Equation 7 below.

[Equation 7]

Here, SR (u) can represent the degree of spatial randomness at pixel X (u).

Y (u) may refer to a set of neighboring pixels surrounding pixel X (u).

R _xy may refer to a cross-correlation matrix between X (u) and Y (u).

R _x ^-1 may be an inverse of a correlation matrix of X (u).

The spatial perception sensitivity calculation unit 220 can numerically grasp the degree of spatial randomness in the input image in units of pixels through Equation (7). In addition, the spatial perception sensitivity calculation unit 220 can determine a flat and regular region having a low degree of randomness in an image through application of a threshold or the like.

The spatial perception sensitivity calculation unit 220 can complete the perceptual sensitivity based map by searching for edge and texture areas existing in the above-mentioned area based on the determined low randomness area.

The spatial perception sensitivity calculation unit 220 may use a difference curvature as shown in Equations (8) and (9) below to determine an area of an edge and a texture of an image.

[Equation 8]

[Equation 9]

Here, G _{x, i} may mean the first-order gradient value of the i-th pixel in the image in the x-axis direction.

G _{y, i} may mean the first-order gradient value of the i-th pixel in the image in the y-axis direction.

Here, G _{xx, i} may mean a secondary gradient value of the i-th pixel in the image in the x-axis direction.

G _{yy, i} may mean a second gradient value in the y-axis direction of the i-th pixel in the image.

G _{xy, i} may mean that a first-order first-order gradient is applied to the first-order gradient value of the i-th pixel in the image in the x-axis direction.

N and E may mean the values of thresholds experimentally derived for defining the region of the image.

IND may refer to an impulse function that gives a value of 1 when the inequality in the function is satisfied.

If the value of r _i for the i-th pixel is determined according to Equation 8, the type of the region of the i-th pixel is determined as one of flat, edge and texture according to the value determined by Equation 9 .

The edge or texture region (i.e., the region of pixels having the value of r ₁ of 1 or 2) determined through Equation 8 has a structure such as an edge and a texture existing on a flat and regular background, This can mean an area that is easy to see from a cognitive point of view.

As described above, the spatial cognition sensitivity calculation unit 220 can identify an area where encoding distortion such as block artifacts is conspicuous according to cognitive viewpoints in the original image, (240).

An area where the coding distortion is noticeable may include an edge or a texture area. As described above, the region where the encoding distortion is noticeable is transmitted from the spatial perception sensitivity calculation unit 220 to the spatial distortion analysis unit 240, and is used for predicting image quality degradation due to spatial distortion.

Figure 7 shows a method for determining the degree of distortion due to block artifacts according to a cognitive view according to an example.

In FIG. 7, a method of deriving the degree of distortion for the HV plane by applying the cognitive sensitivity-based weighting to the degree of distortion of the HV plane has been illustrated.

The spatial distortion analyzer 240 calculates the degree of distortion of the HV plane and uses the information provided from the spatial perception sensitivity calculator 220 to give a larger weight to the distortion in the edge and texture area on the flat background have. This process can more accurately measure the degree of distortion due to block artifacts from a cognitive point of view.

8 shows a spatial and temporal frame of an image according to an example.

Figure 9 shows a row plane of temporal frames according to an example.

As shown in FIG. 9, a temporal frame of an image may have a large row plane and a column plane.

The first row plane may be a plane that collects the first rows of frames as shown in Fig.

The first column plane may be a plane that is the first columns of the frames as shown in Fig.

In order to reduce the amount of computation, the planes can be extracted at specified intervals. R / 8 can indicate a specified interval. For example, R may be the number of frames.

FIG. 10 shows applying a cognitive sensitivity-based weighting to the degree of distortion for a column plane of temporal frames according to an example.

In Fig. 10, a column plane is shown on the left, and a volume of motion weights is shown on the right.

The motion weighted volume may be a frame-by-frame multiplied by a center-biased Gaussian weight. That is to say, the motion weighting volume may be the weighted function of the Gaussian distribution multiplied by the residual image between the adjacent frames described in step 330.

The volume of the thermal plane and the motion weighting can be multiplied by plane-wise. That is to say, the motion weight map for Difference _row1 and Difference _row1 in the column plane can be multiplied. Difference _row1 may be the difference in the first row.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium.

The computer-readable recording medium may include information used in embodiments according to the present invention. For example, the computer readable recording medium may comprise a bit stream, and the bit stream may comprise the information described in embodiments according to the present invention.

The computer-readable recording medium may comprise a non-transitory computer-readable medium.

The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

100: Electronic device
110:
120:
130:
210: image feature information extracting unit
220: Spatial cognitive sensitivity calculation unit
230: temporal perception sensitivity calculation unit
240: Spatial Distortion Analysis Unit
250: temporal distortion analysis unit
260:

Claims (1)

A spatial distortion analyzer for generating spatial distortion information based on the spatial feature information of the original image, the spatial feature information of the comparative image, and the spatial cognitive sensitivity based information of the original image;
A temporal distortion analyzer for generating spatial distortion information based on temporal feature information of the original image, temporal feature information of the comparative image, and temporal cognitive sensitivity based information of the original image; And
An index calculation unit for generating an image quality index for subjective image quality based on the temporal distortion information and the spatial distortion information,
.

Images