KR101717399B1 - The automatic system and method for measuring subjective quality score, recording medium thereof - Google Patents

Abstract

Disclosed are a system and a method for automatically measuring a subjective image quality score and a recording medium thereof. The system comprises: an image output unit which transmits an original image and an evaluation image; a display device which displays the received image; a pupil tracking device which tracks the pupil movement of an observer observing the display device and calculates a gazing region; an input unit which receives the gazing region from the pupil tracking device; an entropy unit which calculates an entropy for the gazing region; and a score calculation unit which calculates an image quality score by using the difference in entropies of the original image and the evaluation image. According to the disclosed system, there are merits in that a subjective image quality score can be automatically measured and the reliability thereof is improved.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a system and method for automatically measuring a subjective image quality score,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system and method for automatic image quality score measurement, and a recording medium therefor, and more particularly, to a system and method for automatically measuring subjective image quality scores and a recording medium therefor.

The conventional image quality score measurement system has mainly adopted the method of directly inquiring from the observer to measure the subjective image quality score on the image. That is, it is a method of watching an image and describing a subjective image quality score for the image. This method consumes a lot of time and observers can increase the image quality score, which causes a problem of low reliability.

In order to solve the above problems, the present invention provides a system and method for automatically measuring a subjective image quality score having an improved reliability that can automatically measure a subjective image quality score, and a recording medium therefor.

According to a first aspect of the present invention, there is provided an image processing apparatus comprising: an image output unit for transmitting an original image and an evaluation image; A display device for outputting the transmitted image; A pupil tracking device for tracking a pupil movement of an observer observing the display device and calculating a gaze area; An input unit for receiving the gaze area from the pupil tracking device; An entropy part for calculating an entropy with respect to the gazing area; And a score calculating unit for calculating an image quality score using the difference between the entropy of the original image and the entropy of the evaluation image.

A quantification unit for quantifying the gaze region; And a weight assigning unit for assigning a weight to the quantized candidate region, wherein the entropy unit calculates an entropy with respect to the weighted candidate region.

Wherein the weighting unit includes: a 2D weighting unit for giving a weight to a quantized gaze area of a 2D image; And a 3D weighting unit for weighting the quantized gaze area of the 3D image.

Wherein the quantification unit quantizes the gaze area using a Gaussian mixture model.

And the 2D weight assigning unit assigns a weight to the quantized gaze area of the 2D image using the Foveation.

And the 3D weighting unit assigns a weight to the quantized gaze region of the 3D image using the fusion region of the pannel.

Wherein the score calculating unit calculates an image quality score using entropy for the original image and entropy for the evaluation image using a Kullback-Leibler divergence (KLD).

According to a second aspect of the present invention, there is provided an image processing apparatus comprising: an image output unit outputting an original image and an evaluation image; An input unit for receiving a gaze area as a result of eyeball tracking of an observer observing the output image; An entropy part for calculating an entropy with respect to the gazing area; And a score calculating unit for calculating an image quality score using the difference between the entropy of the original image and the entropy of the evaluation image.

A quantification unit for quantifying the gaze region; And a weight assigning unit for assigning a weight to the quantized candidate region, wherein the entropy unit calculates an entropy with respect to the weighted candidate region.

Wherein the weighting unit includes: a 2D weighting unit for giving a weight to a quantized gaze area of a 2D image; And a 3D weighting unit for weighting the quantized gaze area of the 3D image.

According to a third embodiment of the present invention, there is also provided a method of generating an image, the method comprising: outputting an original image and an evaluation image; Calculating a gazed area by tracking a pupil movement of an observer observing the output image; Quantifying the gaze area; Weighting the quantified gaze area; Calculating entropy for the weighted gaze area; And calculating an image quality score using the difference between the entropy of the original image and the entropy of the evaluation image.

The weighting step may include: a 2D weighting step of weighting the quantized gaze area of the 2D image; And a 3D weighting step of weighting the quantized gaze area of the 3D image.

Wherein the quantifying step is characterized by quantifying the gaze area using a Gaussian mixture model.

The 2D weighting step may be performed by applying weights to the quantized gaze area of the 2D image using Foveation.

Wherein the 3D weighting step assigns a weight to the quantized gaze area of the 3D image using the fusion area of the pannel.

The calculating of the image quality score may include calculating an image quality score using entropy of the original image and entropy of the evaluation image using a Kullback-Leibler divergence (KLD).

According to a fourth aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for performing the above-described method for automatically measuring the subjective image quality score.

The present invention is advantageous in that it can automatically measure a subjective image quality score and has an improved reliability.

FIG. 1 illustrates a system for automatically measuring a subjective image quality score according to an exemplary embodiment of the present invention.
2 is a block diagram of a subjective image quality score automatic measurement system according to an embodiment of the present invention.
FIG. 3 shows a Smart Eye Pro which is a pupil tracking device 200. FIG.
FIG. 4 is a graphical representation of a gazing area for an image due to a compression distortion phenomenon.
FIG. 5 is a graphical representation of a gazing area for an image according to depth difference.
FIG. 6 illustrates comparison of quantization graphs of a Gaussian model and a Gaussian mixture model.
Figure 7 shows a visual perception model for weighting.
FIG. 8 illustrates results of calculation of entropy H for each of the candidate regions.
FIG. 9 is a graph comparing ToVA, which is an image quality score, by a subjective image quality score automatic measurement system according to an embodiment of the present invention, according to distortion and depth.
FIG. 10 is a flowchart illustrating a method for automatically measuring a subjective image quality score according to an exemplary embodiment of the present invention. Referring to FIG.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 illustrates a system for automatically measuring a subjective image quality score according to an exemplary embodiment of the present invention.

Referring to FIG. 1, a subjective image quality score automatic measurement system according to an exemplary embodiment of the present invention includes a display device 100, a pupil tracking device 200, and a subjective image quality score automatic measuring device 300. In order to measure the image quality score, the original image and the image to be evaluated are selected in the subjective image quality score automatic measuring device 300 and reproduced through the display device 100. The observer observes the original image and the image to be evaluated, which are reproduced in the display device 100. At this time, the pupil tracking device 200 tracks the movement of the observer's pupil. Since the distribution of the gaze area of the observer changes according to the distortion of the image quality, it becomes possible to evaluate the image quality through the distribution of the observer's gaze area. In addition, the gaze area can be tracked by tracking the movement of the observer's pupil. Here, the gaze area may appear differently depending on how often an observer looks at a specific area as an area where the observer is gazing. The tracking information of the pupil tracking device 200 is input to the subjective image quality score automatic measuring device 300, and the subjective image quality score automatic measuring device 300 automatically calculates a subjective image quality score based on the inputted tracking information .

2 is a block diagram of a subjective image quality score automatic measurement system according to an embodiment of the present invention.

Referring to FIG. 2, the subjective image quality score automatic measurement system according to an exemplary embodiment of the present invention mainly includes a display device 100, a pupil tracking device 200, and a subjective image quality score automatic measuring device 300. The subjective image quality score automatic measuring apparatus 300 includes an image output unit 310, an input unit 320, a quantization unit 330, a weight assigning unit 340, an entropy unit 350 and a score calculating unit 360 .

The video output unit 310 reproduces an evaluation video image to be evaluated through the display device 100. In order to evaluate the image quality, a reference original image is required, so that the image output unit 310 can output both the original image and the evaluation image. For example, it is possible to use a method of outputting an original image with a certain frame and outputting an evaluation image with the same frame. At this time, the images to be evaluated may be 2D images or 3D images. Accordingly, the display device 100 may be a device capable of 3D display.

Now, the pupil tracking device 200 tracks the movement of the observer's pupil for each reproduced image. Since the tracking method of the pupil tracking device 200 is a known technique, any of various devices may be used. For example, you could use a pupil tracking device like the Smart Eye Pro.

FIG. 3 shows a Smart Eye Pro which is a pupil tracking device 200. FIG.

Referring to FIG. 3, the Smart Eye Pro, which is a pupil tracking device, can track the pupil movement of the pupil and display the observer's gaze area on the screen. After tracing the movement of the pupil of the observer, the observer's gaze area is calculated through the three-dimensional geometry. Smart Eye Pro also keeps track of people's gaze, allowing them to graphically represent the coverage area.

FIG. 4 is a graphical representation of a gazing area for an image due to a compression distortion phenomenon.

FIG. 5 is a graphical representation of a gazing area for an image according to depth difference.

Referring to FIGS. 4 and 5, it can be seen that various compression distortion phenomena and a change in the image quality due to the depth difference affect the observer's gaze area. Referring to FIG. 4, it can be seen that the degree of change of the gazing region varies depending on the degree of distortion of the distorted image. it can be seen that the variance of the gazing area increases as the distortion degree of the images corresponding to (a) to (c) increases. In the images corresponding to (d) to (e) . Referring to FIG. 5, in all the examples (a) to (d), it can be seen that as the depth difference increases, the variance of the gazing area increases. As described above, although there is a difference in directionality depending on the types of images, it can be seen that as the image quality difference between the full image and the evaluation image becomes larger, the variance of the gaze region becomes larger. Therefore, it is possible to quantify the variance of the gaze region in the original image and the variance of the gaze region in the evaluation image, and compare the differences to calculate the subjective image quality score.

The pupil tracking device 200 transmits the gaze area of the observer to the image through the input unit 320. The quantification unit 330 performs the function of quantizing the gaze area to easily calculate the image quality score.

The quantification unit 330 may quantify the examination area using various methods. Various known techniques can be used, and a method using a Gaussian Mixture Model (GMM fitting) will be described as an example.

FIG. 6 illustrates comparison of quantization graphs of a Gaussian model and a Gaussian mixture model.

Referring to FIG. 6, when a Gaussian model is applied to a gazing region as shown in (a), it is quantified as a single graph as shown in (b). However, when the Gaussian mixture model is used, it is quantified as a number of graphs as shown in (c). In this way, quantification using the Gaussian mixture model can be modeled as a plurality of graphs for each of the clustered regions, so that various distribution patterns can be compared. Therefore, There is an advantage that accurate comparison is possible.

번째 응시영역

의 분포는 하기 수학식으로 정의된다.Using Gaussian mixture model

The second gaze area

Is defined by the following equation.

와

는 응시영역

에서의

축과

축의 분포를 의미하며,

는 초점의 분산값이다. 따라서 응시영역

의 결합분포

는 하기 수학식으로 정의된다.In Equation (1)

Wow

The examination area

In

Axis

Means the distribution of the axis,

Is the variance of the focus. Therefore,

Bond distribution of

Is defined by the following equation.

Therefore, the entire examination area using the Gaussian mixture model can be quantified as shown in the following equation.

은 응시영역의 개수를 의미하며,

는 각 응시영역의 분포에 대한 가중치(

)이다.In Equation (3)

Denotes the number of the gaze area,

Is a weight for the distribution of each of the candidate regions

)to be.

Next, the weight assigning unit 340 assigns a weight to the quantized gaze area. The weight assigning unit 340 may include a 2D weight assigning unit 342 and a 3D weight assigning unit 344.

Figure 7 shows a visual perception model for weighting.

Referring to FIG. 7, the 2D weight assigning unit 342 may weight a 2D image, and various known methods can be used as a weighting method. For example, the Foveation model in (a) can be used. The foveation model is a model of the visual weighting that occurs due to the non-uniform distribution of the retina's photoreceptor cell, which is defined by the following equation.

는 사람이 응시하는 곳과의 거리(

)를 의미하고,

는 초점으로부터 다른 위치

까지의 픽셀 거리를 의미하며,

는 초점으로부터의 각도(Eccentricity) e를 의미한다. 또한,

는 Foveation에서 정의되는 상수값이다. 각각의 수치는

,

와

로 설정될 수 있으며, 한계주파수

에 대하여 하기 수학식과 같이 가중치를 적용할 수 있다.In Equation (4)

Is the distance from where the person stare

), ≪ / RTI >

Lt; RTI ID = 0.0 >

Pixel distance, < RTI ID = 0.0 >

Means the eccentricity e from the focus. Also,

Is a constant value defined in Foveation. Each figure

,

Wow

, And the limit frequency

Can be weighted as shown in the following equation.

는 Foveation이 적용된 지점에서의 공간 한계 주파수이다. 최종적으로 2D 가중치는 모든 응시영역에서 나타내지게 되고, 하기 수학식과 같다.In Equation (5)

Is the spatial limit frequency at the point where the foveation is applied. Finally, the 2D weights are represented in all the candidate regions, and are expressed by the following equations.

Referring to FIG. 7, the 3D weight assigning unit 344 may weight a 3D image. Various known methods can be used as a weighting method. For example, we can use the Panum's Fusional Area model in (b). Referring to the Panum's Fusional Area model, when eyes of both eyes are synthesized at one gaze point in a 3D image, there exists a region in which an object is normally synthesized from the gaze point, The object is not synthesized, and two phenomena (Diplopia) appear. The weight by the Panum's Fusional Area model is defined by the following equation.

는 패넘의 융합 영역의 한계값이며,

는 실험적으로 얻어지는 고정값으로서,

와 같은 값을 가질 수 있다. 또한, 각각의 각도

와

은 응시영역과 주변영역의 양안의 각도를 의미하고, 이 두 각도의 차이를 각도 시차(angular disparity)라고 정의한다. 따라서,

는 초점에서의 각도시차,

은 주변점에서의 각도시차를 의미하며,

이다. 여기서 각도 시차란, 사람이 응시하는 초점으로부터 다른 사물을 인지할 때 양안과 초점과의 각과 다른 사물을 응시하는 각과의 차이를 의미한다. 따라서 두 영역의 각도 차이에 따라 시각적 인지가 달라짐을 모델화하여 3D 영상에 대한 가중치를 산출하여 적용할 수 있다.In Equation (7)

Is the limit value of the fusion region of the pnum,

Is an experimentally obtained fixed value,

Lt; / RTI > Further,

Wow

Refers to the angle of both eyes in the gaze area and the surrounding area, and the difference between these two angles is defined as angular disparity. therefore,

The angular parallax at focus,

Represents the angular parallax at the peripheral points,

to be. Here, the angular disparity means the difference between the angles of the binocular and focal points and the angle of gazing at other objects when recognizing another object from the focus of gazing. Therefore, it is possible to calculate and apply a weight value to a 3D image by modeling that a visual perception is changed according to an angle difference between the two areas.

The entropy unit 350 calculates the entropy for the weighted gaze area.

FIG. 8 illustrates results of calculation of entropy H for each of the candidate regions.

Referring to FIG. 8, the degree of variance of the gazing area can be compared by calculating entropy for each image. The entropy is defined by the following equation.

The entropy can be calculated using Equation (8). Referring to FIG. 7, it can be seen that entropy has a higher value as the crossing region spreads widely.

When the above process is performed on the original image and the evaluation image to calculate the entropy, the score calculation unit 360 finally calculates the image quality score using the difference of the calculated entropy of the original image and the evaluation image. There are also various methods for comparing differences in entropy. For example, it can be calculated using Kullback-Leibler divergence (KLD). KLD is defined by the following equation.

는 원본 영상과 평가 영상의 응시영역 분포의 KLD이고,

은 각각 원본 영상과 평가 영상의 엔트로피를 의미한다.

는 각각 최소 최대 엔트로피를 가지는 값이며, 규격화를 위해 사용되는 값이다. 수학식 9을 이용하여, 최종적으로 화질 점수인 ToVA를 산출해 낼 수 있다.In Equation (9)

Is the KLD of the gaze region distribution of the original image and the evaluation image,

Represents the entropy of the original image and the evaluation image, respectively.

Is a value having a minimum maximum entropy, and is a value used for normalization. Using the equation (9), it is possible to calculate ToVA which is the image quality score finally.

FIG. 9 is a graph comparing ToVA, which is an image quality score, by a subjective image quality score automatic measurement system according to an embodiment of the present invention, according to distortion and depth.

Referring to FIG. 9, it can be seen that as the distortion degree and the depth level (Disparity Level) increase, the ToVA score increases and the image quality decreases.

As described above, the image quality score calculated by the system for automatically measuring the subjective image quality score according to the embodiment of the present invention is a subjective value that can be different depending on the observer, but the image quality calculated using the purely visual fatigue and the gaze area Therefore, it is possible to measure automatically and quickly with only image observation, and it has a merit of high reliability.

The subjective image quality score automatic measuring apparatus 300 according to another embodiment of the present invention includes an image output unit 310, an input unit 320, a quantization unit 330, a weight assigning unit 340, an entropy unit 350, And a calculation unit 360.

Referring to FIG. 2, an apparatus 300 for automatically measuring a subjective image quality score according to another embodiment of the present invention includes a function of automatically measuring a subjective image quality score using the display device 100 and the pupil tracking device 200 . The subjective image quality point automatic measuring apparatus 300 according to another embodiment of the present invention can operate and function in the same manner as described in the embodiment of the present invention.

FIG. 10 is a flowchart illustrating a method for automatically measuring a subjective image quality score according to an exemplary embodiment of the present invention. Referring to FIG.

The method of automatically measuring a subjective image quality score according to an exemplary embodiment of the present invention includes a step S910 of outputting an image, a step S920 of calculating a gaze area, a step S930 of quantizing, a step of weighting S940, Calculating entropy (S950), and calculating an image quality score (S960).

Step S910 of outputting an image is a step of outputting an original image for comparison and an evaluation image for measuring a score through a display device for measuring an image quality score of the image.

 In step S920 of calculating the gaze area, the observer observes the output image. At this time, the pupil tracking device tracks the observer's pupil to calculate the gaze area.

In the step of quantification (S930), various calculated quantification methods are quantified. For example, a Gaussian mixture model can be used.

In the weighting step S940, a weight is given to the quantized gaze area, and various known methods can be used. For example, 2D images can be weighted using the Foveation model, and 3D images can be weighted using the Panum's Fusional Area model.

In step S950 of calculating the entropy, entropy is calculated for the weighted gaze area. The entropy value can be calculated according to the degree of dispersion of the gaze region.

In the step of calculating the image quality score (S960), the final image quality score is calculated using the difference of the entropy values calculated for the original image and the evaluation image. Methods for comparing the values of entropy Various known methods can also be used. For example, image quality scores can be calculated using Kullback-Leibler divergence (KLD).

Embodiments of the present invention may be implemented in the form of program instructions that can be executed on various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Such as magneto-optical, and ROM, RAM, flash memory, etc. Examples of program instructions recorded on the medium include machine code such as those produced by a compiler, Lt; RTI ID = 0.0 > and / or < / RTI > The hardware devices described above may be configured to operate as at least one software module to perform operations of one embodiment of the present invention, and vice versa.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and specific embodiments and drawings. However, it should be understood that the present invention is not limited to the above- Those skilled in the art will appreciate that various modifications and changes may be made thereto without departing from the scope of the present invention. Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

100: display device
200: pupil tracking device
300: Automatic measurement device for subjective image quality score
310: Video output unit
320:
330: Quantification part
340: Whether or not the weight is provided
342: Whether 2D weights are assigned
344: 3D weighting function
350: entropy portion
360: score calculation unit

Claims (17)

An image output unit for transmitting the original image and the evaluation image;
A display device for outputting the transmitted image;
A pupil tracking device for tracking a pupil movement of an observer observing the display device and calculating a gaze area;
An input unit for receiving the gaze area from the pupil tracking device;
A quantification unit for quantifying the gaze region;
A 2D weighting unit for assigning weights to the quantized gaze region of the 2D image;
A 3D weighting unit for weighting the quantized gaze area of the 3D image;
An entropy unit for calculating an entropy with respect to the weighted gaze area; And
And a score calculating unit for calculating an image quality score using a difference between entropy of the original image and entropy of the evaluation image. delete delete The method according to claim 1,
Wherein the quantifying unit quantizes the gazing area using a Gaussian mixture model. 5. The method of claim 4,
Wherein the 2D weighting unit assigns a weight to the quantized gaze area of the 2D image using the Foveation. 6. The method of claim 5,
Wherein the 3D weighting unit assigns a weight to the quantized gaze area of the 3D image using the fusion area of the pannel. The method according to claim 6,
Wherein the score calculating unit calculates an image quality score using entropy of the original image and entropy of the evaluation image using a Kullback-Leibler divergence (KLD). An image output unit for outputting an original image and an evaluation image;
An input unit for receiving a gaze area as a result of eyeball tracking of an observer observing the output image;
A quantification unit for quantifying the gaze region;
A 2D weighting unit for assigning weights to the quantized gaze region of the 2D image;
A 3D weighting unit for weighting the quantized gaze area of the 3D image;
An entropy unit for calculating an entropy with respect to the weighted gaze area; And
And a score calculating unit for calculating an image quality score using the difference between the entropy of the original image and the entropy of the evaluation image. delete delete Outputting an original image and an evaluation image;
Calculating a gazed area by tracking a pupil movement of an observer observing the output image;
Quantifying the gaze area;
A 2D weighting step of weighting a quantized gaze area of a 2D image;
A 3D weighting step of weighting the quantized gaze area of the 3D image;
Calculating entropy for the weighted gaze area; And
And calculating an image quality score using the difference between the entropy of the original image and the entropy of the evaluation image. delete 12. The method of claim 11,
Wherein the quantifying step quantifies the gaze area using a Gaussian mixture model. 14. The method of claim 13,
Wherein the 2D weighting step assigns a weight to the quantized gaze area of the 2D image using Foveation. 15. The method of claim 14,
Wherein the 3D weighting step assigns a weight to the quantized gaze area of the 3D image using the fused area of the Pannum. 16. The method of claim 15,
Wherein the step of calculating the image quality score comprises calculating an image quality score using a Kullback-Leibler divergence (KLD) for the entropy of the original image and the entropy of the evaluation image. A computer-readable recording medium having recorded thereon a program for performing a method of automatically measuring a subjective image quality score according to any one of claims 11 to 13.

Images