KR101649188B1 - Method of measuring 3d effect perception and apparatus for measuring 3d effect perception - Google Patents

Abstract

Provided are a method for measuring three-dimensional effect and an apparatus therefor. The method for measuring three-dimensional effect includes the following steps of: measuring visual attention score (VAS) for an analyzed frame among a plurality of frames included in a three-dimensional image file; and measuring three-dimensional effect for the three-dimensional image file based on the VAS.

Description

TECHNICAL FIELD [0001] The present invention relates to a 3D effect measurement method and apparatus,

The present invention relates to a method and apparatus for measuring a three-dimensional sensation, and more particularly, to a method and apparatus for measuring a three-dimensional sensation using a visual attention score (VAS) to measure a sensation in a three-dimensional image file.

A 3D image is an image that can detect the perspective of an object in a plane on a plane including information on height and width as well as depth. Recently, interest in 3D movies or 3D TVs has been heightened, and various 3D image contents have been produced and distributed. Therefore, researches on methods of producing 3D images and evaluation methods have been actively conducted. When a 3D image is produced, it is necessary for viewers to produce 3D images in a more accurate manner. In other words, since the 3D image itself is an image produced to convey the 3D sensation to the user differently from the 2D image, whether or not the user can feel the 3D sensation well in the 3D image, in other words, It is very important to measure whether or not it is.

Conventionally, research on using visual fatigue as an evaluation criterion of three-dimensional image has progressed a lot, but research on the use of three-dimensional feeling as a criterion for evaluation of a three-dimensional image has not been done much. With respect to the conventional evaluation of the three-dimensional feeling, there is a method of measuring the quality of the three-dimensional feeling using the histogram of the depth image. However, the method has a problem that it is not suitable for the evaluation of the three-dimensional feeling actually felt by the user.

Therefore, it has been required to develop a method and apparatus that can more accurately measure the three-dimensional effect on the three-dimensional image.

[Related Technical Literature]

1. 3D Image Processing Apparatus and Method for Adjusting the Three Dimensional Sensitivity (Patent Application No. 10-2011-0036973)

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a method and apparatus for measuring a three-dimensional sensation, which can more accurately evaluate a three-dimensional sensation by using a visual attention degree when evaluating a three-dimensional sensation.

Another problem to be solved by the present invention is to use a velocity (V) value for each pixel, a distance from the screen (DFS) value and a defocus blur amount (DBA) And a method and apparatus for measuring a three-dimensional sensation using a more accurate visual attention degree calculated by reflecting characteristics of a three-dimensional image.

The problems of the present invention are not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method of measuring a three-dimensional image, the method comprising: calculating a visual attention score (VAS) for a frame to be analyzed among a plurality of frames included in a three- And measuring the stereoscopic effect on the three-dimensional image file based on the stereoscopic view and the stereoscopic view.

According to another aspect of the present invention, the step of measuring a three-dimensional sensation includes a step of weighting a gradient of a depth image of each pixel of a frame to be analyzed, do.

According to another aspect of the present invention, a method of measuring a three-dimensional sensation further includes calculating a visual attention probability (VAP) having a value of 0 to 1 by normalizing the visual attention degree.

According to still another aspect of the present invention, the step of calculating the visual attention probability includes:

, Wherein i is a longitudinal direction index of the pixel and j is a lateral direction index of the pixel.

According to another aspect of the present invention, the step of measuring the three-

Calculating a weighted gradient magnitude by the VAP (WGMVAP) by the visual attention probability, and measuring a 3D sensation on the 3D image file based on the WGMVAP, wherein the VSK is a visual sensitivity kernel (visual sensitivity kernel)

Is the size of the gradient of the depth image of the pixel whose coordinates are (x, y), i and x are the longitudinal index of the pixel, and j and y are the lateral index of the pixel.

According to still another aspect of the present invention, the visual sensitivity kernel has a shape that linearly decreases in a horizontal direction and a vertical direction from a point where a time is concentrated.

According to another aspect of the present invention, the step of calculating the degree of visual attention on a frame to be analyzed may include comparing a frame to be analyzed and a previous frame of the frame to be analyzed among a plurality of frames, Generating a velocity (V) value, generating a distance from the screen (DFS) value for each pixel by analyzing a left eye image and a right eye image of a frame to be analyzed, Generating a defocus blur amount (DBA) value for each of the pixels by comparing the blurred image of the frame to be analyzed, generating a defocus blur amount (DBA) value for each of the pixels based on the V value, the DFS value, Wherein the DFS value includes a point at which the stereoscopic image is formed by the left eye image and the right eye image, Characterized in that the rigap.

According to another aspect of the present invention, the visual attention degree for each pixel is larger as the V value and the DFS value are larger and the DBA value is smaller.

According to still another aspect of the present invention, the step of calculating the visual attention degree for each of the pixels is a step of calculating a visual attention degree for each of the pixels by reflecting an interaction between the V value and the DBA value .

According to still another aspect of the present invention, the step of generating a DBA value includes: generating a DBA value for each edge pixel included in an analysis target frame by comparing an analysis target frame and a blurred image; And generating a DBA value for each of the pixels based on the DBA value and the DFS value for each of the included edge pixels.

According to still another aspect of the present invention, the step of generating a DBA value for each of the edge pixels included in the frame to be analyzed includes a step of generating a first gradient that is a gradient of a pixel value of an edge pixel included in a frame to be analyzed Calculating a second gradient, which is a gradient of pixel values of edge pixels included in the blurred image, and a second gradient, which is a gradient of pixel values of edge pixels included in the blurred image, Generating a DBA value for each of the edge pixels included in the frame to be analyzed based on the ratio of the first gradient to the second gradient and the standard deviation of the Gaussian filter.

According to another aspect of the present invention, the step of generating a DBA value for each of all pixels based on a DBA value and a DFS value for each edge pixel included in an analysis target frame includes: A step of extracting a reference area which is an area having the smallest average of DBA values by calculating the average of DBA values for edge pixels included in each of the plurality of areas, And generating a DBA value based on the distance between the reference area and each of the pixels.

According to an aspect of the present invention, there is provided an apparatus for measuring a three-dimensional image, the apparatus comprising: a calculation unit configured to calculate a visual attention score (VAS) for a frame to be analyzed among a plurality of frames included in a three- And a stereoscopic effect measuring unit configured to measure a stereoscopic effect on the three-dimensional image file based on the constructed visual attention degree calculating unit and the visual attention degree.

According to an aspect of the present invention, there is provided a computer-readable medium for calculating a visual attention score (VAS) for a frame to be analyzed among a plurality of frames included in a 3D image file And a set of instructions for measuring a stereoscopic effect on the three-dimensional image file based on the visual attention degree.

The details of other embodiments are included in the detailed description and drawings.

The present invention can more accurately evaluate the stereoscopic effect by using the visual attention in evaluating the stereoscopic effect of a three-dimensional image.

In addition, the present invention can evaluate the 3D effect using a more accurate visual attitude calculated by reflecting the characteristic of the 3D image by using the V value, the DFS value and the DBA value for each pixel.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the specification.

1 is a schematic block diagram of an apparatus for measuring a three-dimensional sensation according to an embodiment of the present invention.
2 is a flowchart illustrating a method of measuring a three-dimensional sensation according to an exemplary embodiment of the present invention.
3 is a graph illustrating a visual sensitivity kernel (VSK) used in the method of measuring three-dimensional sensation according to an embodiment of the present invention.
FIG. 4 is a table describing an experimental environment of an experiment for deriving visual attention in the method of measuring three-dimensional sensation according to an embodiment of the present invention.
FIG. 5 is a table showing a relationship between a V value, a DFS value, and a DBA value and a level in an experiment for deriving visual attention in a method of measuring a three-dimensional sensation according to an embodiment of the present invention.
FIG. 6 is a graph showing data points used in an experiment to derive visual attention in the method of measuring three-dimensional sensation according to an embodiment of the present invention.
FIG. 7 is a table showing the result of the analysis of variance of the visual attention degree calculated through the experiment for deriving the visual attention degree in the three-dimensional sensibility measurement method according to the embodiment of the present invention.
FIG. 8 is a graph illustrating the visual attention degree calculated by the three-dimensional sensing method according to an embodiment of the present invention in accordance with a change in the DFS value and the DBA value.
FIG. 9 is a graph showing the visual attention degree calculated by the three-dimensional sensing method according to an embodiment of the present invention in accordance with the change of the V value and the DBA value.
FIG. 10 is a graph showing the visual attention degree calculated by the three-dimensional sensory measurement method according to an embodiment of the present invention in accordance with the change of the V value and the DFS value.
11A to 11E illustrate a left eye image of a frame to be analyzed and a relative magnitude of a calculated V value, a DFS value, a DBA value, and a visual attention degree, which are analyzed through a three-dimensional sensing method according to an embodiment of the present invention.
12A and 12B are scatter diagrams of a method for measuring a three-dimensional sensation, a comparison example, and a subjective evaluation result according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

Like reference numerals refer to like elements throughout the specification.

Each block of the accompanying block diagrams and combinations of the steps of the flowcharts may be performed by algorithms or computer program instructions comprised of firmware, software, or hardware. These algorithms or computer program instructions may be embedded in a processor of a general purpose computer, special purpose computer, or other programmable digital signal processing device, so that the instructions that are executed by a processor of a computer or other programmable data processing apparatus Generate means for performing the functions described in each block or flowchart of the block diagram. These algorithms or computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement a function in a particular manner, It is also possible for instructions stored in a possible memory to produce a manufacturing item containing instruction means for performing the function described in each block or flowchart of each block diagram. Computer program instructions may also be stored on a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible that the instructions that perform the processing equipment provide the steps for executing the functions described in each block of the block diagram and at each step of the flowchart.

Also, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function (s). It should also be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order according to the corresponding function.

It is to be understood that each of the features of the various embodiments of the present invention may be combined or combined with each other partially or entirely and technically various interlocking and driving is possible as will be appreciated by those skilled in the art, It may be possible to cooperate with each other in association.

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

1 is a schematic block diagram of an apparatus for measuring a three-dimensional sensation according to an embodiment of the present invention. 2 is a flowchart illustrating a method of measuring a three-dimensional sensation according to an exemplary embodiment of the present invention. 1, a cubic intensity measuring apparatus 100 includes a visual attention degree calculating unit 110 and a cubic intensity measuring unit 120. The visual attention degree calculating unit 110 includes a V value generating unit 111, a DFS Value generating unit 112, a DBA value generating unit 113, and a calculating unit 114. [

First, the visual attention degree calculating unit 110 calculates a visual attention degree of an analysis object frame among a plurality of frames included in the 3D image file (S210).

In order to measure a three-dimensional sensation of a three-dimensional image file, which will be described later, the apparatus 100 for measuring a three-dimensional sensation according to an exemplary embodiment of the present invention first calculates a visual attention degree of an analysis target frame among a plurality of frames included in a three- . Hereinafter, a method for calculating the visual attention degree that can be used in the three-dimensional sensing apparatus 100 according to an embodiment of the present invention will be described in detail.

A three-dimensional image file analyzed by the three-dimensional sensing apparatus 100 according to an embodiment of the present invention includes a plurality of frames, and each of the plurality of frames includes a left-eye image and a right-eye image. The V-value generating unit 111 of the visual attention degree calculating unit 110 compares the analysis target frame, which is a frame to be analyzed, among the plurality of frames with a previous frame that is a temporally preceding frame with respect to the analysis target frame.

The V value generation unit 111 compares pixel values of all the pixels of all frames of the analysis target frame with pixel values of all pixels of the previous frame and extracts pixels of the analysis object frame having the same pixel value and pixels of the previous frame . Also, the V value generation unit 111 may recognize the same pixel even if the pixel value is not exactly the same, even if the difference of the pixel value is, for example, within 1%. In addition, the V-value generation unit 111 may determine whether the pixels are the same pixel by grouping a predetermined number of pixels. In this case, the V-value generation unit 111 regards the pixels as the same when the sum of the pixel errors in the block is less than or equal to the reference value. The V-value generation unit 111 can recognize both of the pixels of the analysis target frame and the pixels of the previous frame as the same pixel when they are positioned within a predetermined distance. For example, the V-value generation unit 111 may recognize that the pixels are the same pixel only when the distance between the pixels of the analysis target frame and the pixels of the previous frame is within 64 pixels.

The V value means the displacement per unit time of each pixel in the three-dimensional image. For example, if a 3-D image file contains 24 frames per second and the object displayed on the image moves by 1 pixel for 1/24 second, the V value of the object-displayed pixel may be 24 pixels / sec.

The V value generation unit 111 generates a V value by extracting the pixels of the analysis object frame having the same pixel value and the pixels of the previous frame to calculate the distance between them and multiplying the calculated distance by the number of frames per second . The V value generation unit 111 may store the generated V value in a memory.

Next, the DFS value generation unit 112 of the visual attention degree calculation unit 110 generates a DFS value for each of all the pixels by analyzing the left eye image and the right eye image of the analysis target frame.

Since the frame to be analyzed is a three-dimensional image, it includes a left-eye image and a right-eye image. The viewer feels a three-dimensional feeling due to the difference between the left-eye image and the right-eye image.

The DFS value generation unit 112 may extract a left eye image and a right eye image from the analysis target frame. The DFS value generating unit 112 can calculate a point at which a stereoscopic image is formed on the basis of a preset virtual viewer based on a left eye image and a right eye image. Specifically, the DFS value generation unit 112 may calculate a straight line between a specific pixel of the left eye image and the left eye of the virtual viewer. The DFS value generation unit 112 may calculate a straight line between the pixel of the right eye image corresponding to the specific pixel of the left eye image and the right eye of the virtual viewer. The DFS value generation unit 112 can calculate a point at which a three-dimensional image is formed by calculating a point where two straight lines intersect. The DFS value means a vertical distance between a point at which a stereoscopic image is formed and a screen, that is, a depth, by the left eye image and the right eye image calculated in the above-described manner. The DFS value generation unit 112 may generate the DFS value by calculating the distance between the point at which the stereoscopic image is formed and the screen. The DFS value can be positive if the point at which the stereoscopic image is located is negative if it is farther away from the viewer than it is when it is closer to the screen.

Next, the DBA value generating unit 113 of the visual attention degree calculating unit 110 generates a DBA value for each of all the pixels by comparing the blurred image of the analysis target frame and the analysis target frame.

The blurred image can be generated by blurring the frame to be analyzed. The DBA value generation unit 113 may generate a blurred image by synthesizing a function for blurring processing on a function representing a pixel value for each pixel of the analysis object frame.

The DBA value generation unit 113 can generate a DBA value for each of the edge pixels included in the analysis target frame by comparing the analysis target frame and the blurred image. The DBA value means a value indicating the degree of blur of a pixel. The DBA value is small for pixels in focus, and large for pixels in out of focus.

Specifically, the DBA value generator 113 may calculate a first gradient which is a gradient of a pixel value of an edge pixel included in a frame to be analyzed. An edge pixel means a pixel whose pixel value changes abruptly in comparison with the pixel value of a neighboring pixel neighboring the pixel. For example, an edge pixel can be defined as a pixel having a gradient of a pixel value of a certain level or more. The gradient of the pixel value of the edge pixel means the rate of change of the pixel value between the edge pixel and the neighboring pixel of the edge pixel. The DBA value generation unit 113 may calculate the first gradient based on the pixel value of the edge pixel, the pixel value of the pixel neighboring the edge pixel, and the distance between the edge pixel and the pixel neighboring the edge pixel.

The DBA value generation unit 113 may generate a blurred image by convoluting the frame to be analyzed and the Gaussian filter. The Gaussian filter means a two-dimensional Gaussian function used for blurring an image. The characteristics of the Gaussian filter are determined by the standard deviation of the Gaussian filter. An exemplary equation for representing a Gaussian filter is as follows.

[Equation 1]

Where G (x, y) is a Gaussian filter, i.e., a two-dimensional Gaussian function, and sigma is a standard deviation of a Gaussian filter.

The DBA value generation unit 113 can generate a blurred image corresponding to the analysis object frame by convoluting the function indicating the pixel value of the analysis object frame and the two-dimensional Gaussian function. The larger the standard deviation of the two-dimensional Gaussian function, the more blurred the image is produced.

The DBA value generation unit 113 may calculate a second gradient which is a gradient of pixel values in edge pixels included in the blurred image. The DBA value generation unit 113 may calculate the second gradient by applying the same method as that of the first gradient to the blurred image.

The DBA value generation unit 113 may generate a DBA value for each of the edge pixels included in the analysis target frame based on the ratio of the size of the first gradient to the size of the second gradient and the standard deviation of the Gaussian filter. The DBA value generation unit 113 generates a DBA value that becomes smaller as the size of the first gradient with respect to the size of the second gradient increases, that is, as the analysis target frame becomes clearer. An exemplary equation for generating a DBA value for each of the edge pixels included in the frame to be analyzed is as follows.

&Quot; (2) "

Here, i ₁ (x, y) is a function representing the pixel value for the coordinates of the edge pixel of the image to be analyzed, and i ₂ (x, y) represents a pixel value for the coordinates of the edge pixel of the blurred image. And sigma is the standard deviation of the two-dimensional Gaussian function.

The DBA value generation unit 113 may generate a DBA value for each of all the pixels based on the DBA value and the DFS value for each edge pixel included in the analysis target frame.

Specifically, the DBA value generation unit 113 can divide the analysis target frame into a plurality of areas based on the DFS value. For example, the DBA value generation unit 113 may generate a DBA value including a first region including pixels having a DFS value of 0m to 0.2m, a second region including pixels having a DFS value of 0.2m to 0.4m, And a third region including pixels having a DFS value of 0.4 m to 0.6 m.

The DBA value generation unit 113 can extract the reference area which is the area having the smallest average value of the DBA values by calculating the average of the DBA values for the edge pixels included in each of the plurality of areas. For example, when the frame to be analyzed is divided into the first area, the second area, and the third area, the DBA value generation unit 113 generates the DBA value of the edge pixels included in the first area, the second area, Values of the first region, the second region, and the third region, the region having the lowest average value of the DBA values can be extracted as the reference region. A region having a low DBA value means a relatively clear region, that is, a region in the background state.

The DBA value generation unit 113 may generate a DBA value for all the pixels based on the average of the DBA values of the reference region and the DFS value difference between the pixels of the reference region and each of all the pixels. Since the DBA value generation unit 113 shows a large degree of blurring of a pixel having a large difference between the pixel of the reference area and the DFS value, which is the area in the background, the average value of the DBA values of the edge pixels included in the reference area is The DBA value for the entire pixel can be generated so that a pixel having a large difference between the pixels included in the reference area and the DFS value has a larger DBA value. For example, when the frame to be analyzed is divided into the first area, the second area and the third area and the second area is extracted as the reference area, the DBA value generator 113 generates the DBA value of the edge pixel included in the second area The DBA value for all the pixels can be generated so that a pixel having a large difference between the DFS value of 0.3m and the DFS value of the second area has a larger DBA value based on the average of the DBA values. The DBA value for each of all the pixels may be determined by a previously stored function having a difference between the DBA value and the DFS value of the edge pixel included in the reference area as a variable.

As described above, the DBA value of the edge pixel is preferentially calculated without directly calculating the DBA value of all the pixels, and then the DBA value of the entire pixel is calculated by using the difference between the DBA value of the edge pixel and the DFS value with the edge pixel The time for calculation can be shortened.

According to some embodiments, the units of the V value, the DFS value, and the DBA value may be converted into the same unit. For example, when the unit of the V value generated by the generation unit 110 is pixel / sec, the unit of the DFS value is m, and the unit of the DBA value is pixel, the unit of the V value is converted into m / sec , And the unit of the DBA value can be converted to m.

Next, the calculation unit 114 calculates the visual attention degree for each of all the pixels based on the V value, the DFS value, and the DBA value. The calculation unit 114 can calculate the visual attention degree through a previously stored function having the V value, the DFS value, and the DBA value as variables.

The visual attention degree may have a larger value as the V value and the DFS value are larger and the DVA value is smaller. That is, as the speed of the object displayed on the three-dimensional image is fast and the distance from the viewer is close to the viewer, the more the viewer is concentrated on the object, the higher. Accordingly, the calculation unit 114 can calculate the visual attention degree such that the larger the V value and the DFS value are, and the larger the DBA value, the greater the value.

The calculation unit 114 may calculate the visual attention degree by reflecting the interaction between the V value and the DBA value. When the speed of the object displayed on the three-dimensional image is high, it is difficult for the viewer to perceive the degree of blur, so that the degree of blur of the object does not greatly affect the concentration of the viewer. On the other hand, when the speed of the object displayed on the three-dimensional image is low, the observer can easily recognize the degree of blurring, so the degree of blurring of the object greatly affects the viewer's concentration. Accordingly, the calculation unit 114 can calculate the visual attention degree by reflecting the interplay between the V value and the DBA value so that the influence of the DBA value becomes smaller as the V value becomes larger and the influence of the DBA value becomes larger as the V value becomes smaller have.

An exemplary equation for calculating the visual attention based on the V value, the DFS value, and the DBA value is as follows.

&Quot; (3) "

Here, a, b ₁ , b ₂ , c, d, e, and k may be predetermined constants. Preferably, a is about 202901.342, b ₁ is about -1677.090, b ₂ is about 0.609, c is about -0.003, d is about -146448.366, e is about 10.412, and k is about 0.080. The term " about " can be interpreted to include values up to < RTI ID = 0.0 > 10% < / RTI >

According to some embodiments, the calculation unit 114 may calculate a visual attention probability (VAP) having a value of 0 to 1 by normalizing the visual attention degree. The calculation unit 114 can normalize the visual attention degree so that the relative size of the visual attention degree for a specific pixel among all the pixels included in the analysis object frame can be easily grasped. An exemplary equation for calculating the visual attention probability is as follows.

&Quot; (4) "

VAS (i, j) is a visual attention degree of a pixel whose coordinates are (i, j), and VAP (i, j) is a coordinate of a pixel Is the visual attention probability of a pixel that is (i, j). The height is the height of the frame to be analyzed, and the width is the width of the frame to be analyzed.

As described above, by calculating the degree of visual attention using various characteristic values that can be extracted from the three-dimensional image file such as the V value, the DFS value, and the DBA value, the user's concentration on each of the three- A visual attention degree that can be expressed can be calculated.

Next, the three-dimensional sensing unit 120 measures the three-dimensional sensibility of the three-dimensional image file based on the visual attention degree (S220).

The three-dimensional sensibility measurement unit 120 measures the three-dimensional sensibility of the three-dimensional image file based on the visual attention degree calculated by the visual attention degree calculation unit 110. [ Specifically, the three-dimensional sensing unit 120 measures a three-dimensional sensation in such a manner that the weight of the depth image of each pixel of the frame to be analyzed is weighted according to the degree of visual attention. An exemplary mathematical expression for measuring the cubic effect is as follows

&Quot; (5) "

Here, the weighted gradient magnitude by the VAP (WGMVAP) represents the weight of the gradient weighted by the visual attention probability. The larger the WGBVAP value, the better the 3D effect of the frame to be analyzed. Also, x is the vertical coordinate of the pixel, y is the horizontal coordinate of the pixel, h is the height of the frame to be analyzed, and w is the width of the frame to be analyzed.

Is the magnitude of the gradient of the depth image of the pixel whose coordinates are (x, y), VAP is the visual attention probability as described above, and VSK is the visual sensitivity kernel. W (x, y) denotes a weight function for weighting the size of the gradient of the depth image of each pixel, and w (x, y) denotes a convolution operation of VAP and VSK. The weight function can be expressed in more detail by the following equation.

&Quot; (6) "

Here, VAP (i, j) is the visual attention probability of the pixel whose coordinates are (i, j) as described above.

In the 3D method and apparatus 100 according to the embodiment of the present invention, the viewer can view the three-dimensional image based on the fact that the viewpoint is concentrated in the three-dimensional image, that is, , And the stereoscopic effect that viewers perceive is likely to be influenced by the depth information around the point of view. Therefore, the weights of the depth slope magnitudes in the pixels around the center of time are reflected by using the visual sensitivity kernel. Hereinafter, the visual sensitivity kernel will be described in detail.

FIG. 3 is a graph showing a visual sensitivity kernel used in the method of measuring three-dimensional sensation according to an embodiment of the present invention.

Referring to FIG. 3, in the method and apparatus for measuring three-dimensional sensation 100 according to an exemplary embodiment of the present invention, focusing on an object at a specific distance will cause a decrease in visual sensitivity to an object at a different distance. The shape of the kernel is linearly reduced from the point where the vision is concentrated to the horizontal and vertical directions. In other words, the visual sensitivity kernel gives the largest weight to the size of the gradient of the depth-wise pixels of the time-concentrated pixels, that is, the pixels of high vision probability, and the pixels farther away from the time- So that the weight given to the size of the image is reduced.

The viewer does not perceive the frame to be analyzed on a pixel basis, so that not only the pixel with time-concentration but also the pixel around the pixel with time-concentration can be intensively recognized by the viewer. Therefore, in the method and apparatus for measuring three-dimensional sensibility 100 according to the embodiment of the present invention, the visual sensitivity kernel is configured to have a shape that linearly decreases in the horizontal and vertical directions from the point where the time is concentrated, It is possible to more accurately measure the stereoscopic effect perceived by the viewers by using the weight function by the convolution operation of the weight function.

That is, in the method and apparatus for measuring three-dimensional sensations 100 according to the embodiment of the present invention, not only the dimensionality of a frame to be analyzed is measured using only the size of a gradient of a depth image of a pixel, It is possible to more accurately measure the three-dimensional effect on the frame to be analyzed by weighting the size of the gradient of the depth image of the high pixel and the surrounding pixels.

FIG. 4 is a table describing an experimental environment of an experiment for deriving visual attention in the method of measuring three-dimensional sensation according to an embodiment of the present invention. FIG. 5 is a table showing the relationship between the V value, the DFS value, and the DBA value and the level in the experiment for deriving the visual attention in the three-dimensional sensory measurement method according to the embodiment of the present invention. FIG. 6 is a graph showing data points used in an experiment for deriving visual attention in a method of measuring a three-dimensional sensation according to an embodiment of the present invention.

Experiments to derive the method of calculating visual attention according to an embodiment of the present invention were performed on 30 subjects who had no problems in viewing three-dimensional images. The mean age of the subjects was 23.0 years, with a visual acuity of 0.6 or more and a difference in visual acuity of 0.3 or less. Referring to FIG. 4, the resolution of the 3D image used in the experiment is 1920 × 1080, the number of frames per second is 24 fps, the aspect ratio of the screen used in the experiment is 16: 9, the size of the screen is 46 inches, The distance between the subjects is 3 m.

The experiment proceeds as follows. Let the subject show the image of two fish passing by and then let the subject choose one of the two fish to be gaze. The fish included in the image has a level V value, a DFS value, and a DBA value corresponding to the fifteen data points shown in Fig. Referring to FIG. 5, for example, the V value of a fish corresponding to a data point of (0, 0, 0) is 0.202, the DFS value is 0, and the DBA value is 0.126. The subjects watch 105 images including two kinds of fish among 15 kinds of fish, and then, the fish whose eyes are concentrated among two kinds of fish are selected. The number of times each of the fifteen kinds of fish is selected by the subject is measured as the visual attention degree for the data point corresponding to each of the fifteen kinds of fish. For example, if the number of fish selected for a data point of (0, 0, 0) is 10, then the visual attention to the data point of (0, 0, 0) is 10.

FIG. 7 is a table showing the result of the analysis of variance of the visual attention degree calculated through the experiment for deriving the visual attention degree in the three-dimensional sensibility measurement method according to the embodiment of the present invention.

Referring to FIG. 7, when the variance analysis is performed to determine the significance of the V value, the DFS value, and the DBA value for the visual attention degree calculated by the above-described experiment, Value was less than 0.001 and the p-value for the DFS value was calculated to be 0.050. That is, when the significance level is 5%, there is a result that there is significance of the V value, the DFS value and the DBA value with respect to the visual attention degree. Also, when the significance level is 5%, there is a result that there is significance of the interplay between the V value and the DBA value for visual attention.

Through the above-described experiment and analysis of variance, it can be seen that the interaction between the V value, the DFS value, the DBA value and the V value and the DBA value is statistically significant for the visual attention degree, and the V value, the DFS value and the DBA value , It is possible to model [Expression 3] that expresses visual attention to the V value, the DFS value and the DBA value through the central synthetic design method and the regression analysis.

FIG. 8 is a graph illustrating the visual attention degree calculated by the three-dimensional sensing method according to an embodiment of the present invention in accordance with a change in the DFS value and the DBA value.

Referring to FIG. 8, the visual attention degree linearly increases as the DFS value increases. Also, the visual attention degree decreases as the DBA value increases, by drawing the parabolic trajectory. On the other hand, when the V value increases, the rate of change of the visual attention degree according to the DBA value decreases, and the visual attention degree linearly decreases as the DBA value increases.

FIG. 9 is a graph showing the visual attention degree calculated by the three-dimensional sensing method according to an embodiment of the present invention in accordance with the change of the V value and the DBA value.

Referring to FIG. 9, as the DBA value increases, the visual attention degree decreases along the parabolic trajectory. In addition, the visual attention degree increases as the V value increases, drawing the hyperbolic trajectory. On the other hand, the change in the DFS value does not affect the relationship between the V value and the DBA value and the visual attention degree.

FIG. 10 is a graph showing the visual attention degree calculated by the three-dimensional sensory measurement method according to an embodiment of the present invention in accordance with the change of the V value and the DFS value.

Referring to FIG. 10, the visual attention degree increases as the V value increases, drawing the hyperbolic trajectory. In addition, visual attention increases linearly with increasing DFS value. On the other hand, when the DBA value increases, the rate of change of the visual attention degree according to the V value increases.

11A to 11E illustrate a left eye image of a frame to be analyzed and a relative magnitude of a calculated V value, a DFS value, a DBA value, and a visual attention degree, which are analyzed through a three-dimensional sensing method according to an embodiment of the present invention. In FIGS. 11B to 11E, a pixel having a high V value, a DFS value, a DBA value, and a high degree of visual attention is shown as white and a pixel having a low visibility is shown as black.

Referring to FIG. 11A, a frame to be analyzed through the method of calculating the degree of visual attention according to an embodiment of the present invention includes two persons and a background. 11A, only the left eye image of the analysis target frame is shown, but the analysis target frame is a three-dimensional image. The analysis object frame shown in FIG. 11A is a frame constituting a scene where a left person and a right person communicate.

Referring to FIG. 11B, since the analysis subject frame corresponds to a scene in which the left character is uttered, the V value of the lip surrounding region of the left character in the speech among the pixels included in the analysis subject frame is relatively higher than the V value of the other region .

11C, since the left character is located in front of the background and the right character is located in front of the left character, the DFS value of the region in which the left character is displayed is higher than the DFS value of the background region, The DFS value of the region where the right character is displayed is generated higher than the DFS value.

Referring to FIG. 11D, since the area in which the left character is displayed is in the background and the area in which the background area and the right character are displayed is in the defocused state, the DBA value of the area in which the left character is displayed is relatively lower than the DBA value of the other area do.

Referring to FIG. 11E, the visual attention degree of the region in which the left figure is displayed, which is the region where the V value and the DFS value are large and the DBA value is low, is calculated to be large. Particularly, Is the largest.

12A and 12B are scatter diagrams of a method for measuring a three-dimensional sensation, a comparison example, and a subjective evaluation result according to an embodiment of the present invention. Specifically, FIG. 12A is a scatter diagram for a comparative example using a mean of a gradient magnitude (GMA) of a depth image, which is a conventional method of measuring a three-dimensional sensation, FIG. 12B is a scatter diagram to be.

In order to evaluate the performance of the three-dimensional sensory measurement method and the comparative example according to the embodiment of the present invention, subjective evaluation results were compared with results obtained by each measurement method.

First, in order to perform a subjective evaluation on subjects, subjective evaluation experiments were conducted on 35 subjects who had no problem viewing 3D images. The average age of the subjects was 23.1, the visual acuity of the subjects was 0.6 or more, The subject's difference in binocular vision was less than 0.3. Subjects were asked to evaluate the sensation of the three-dimensional motion of 26 moving images of a length of 1 second to 2 seconds. The stereoscopic effect was evaluated by the scores of 1 (very small), 2 (small), 3 (normal), 4 (large) and 5 (very large) The reference image was shown. The stereoscopic effect was evaluated based on the last part of each video. The video was shot with a Samsung 2D / 3D 45mm lens attached to the Samsung NX300M camera.

In addition, the results of the motion images for each moving image were calculated by applying the stereoscopic effect measuring method and the comparative example according to the embodiment of the present invention to 26 moving pictures evaluated by the subjects.

12A and 12B are scatter diagrams using regression analysis, in which the horizontal axis represents the average value of the subjective evaluation scores, and the vertical axis represents the results obtained by the three-dimensional feeling measurement method according to the comparative example or one embodiment of the present invention. The optimal linear function of the comparative example shown in FIG. 12A is y = 2.60 * 10 ^-2 x - 1.43 * 10 ^-2 when the straight line is approximated to each dispersion, and the coefficient of determination (R ² ) 0.628. In addition, the optimal linear function in the method of measuring three-dimensional sensation according to an embodiment of the present invention shown in FIG. 12B is y = 1.60 * 10 ^-4 x - 1.22 * 10 ^-4 and the coefficient of determination is 0.688. It is confirmed that the performance of the method of measuring three-dimensional sensation according to an embodiment of the present invention for measuring three-dimensional sensibility on the basis of visual attention is improved compared to a comparative example using merely the average of the magnitudes of the tilt of the depth image.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the present invention is not limited to the disclosed exemplary embodiments, but various changes and modifications may be made without departing from the spirit and scope of the invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the scope of the present invention but to limit the scope of the technical idea of the present invention. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100:
110: visual attention degree calculating unit
111: V-value generating unit
112: DFS value generation unit
113: DBA value generation unit
114:
120:

Claims (14)

Calculating a visual attention score (VAS) for a frame to be analyzed among a plurality of frames included in the 3D image file; And
Measuring a stereoscopic effect on the 3D image file based on the visual attention degree,
Wherein the step of calculating the visual attention degree of the analysis target frame comprises:
Generating a velocity (V) value for each of the pixels included in the analysis target frame by comparing the analysis target frame and a previous frame of the analysis target frame among the plurality of frames;
Generating a distance from the screen (DFS) value for each of the pixels by analyzing a left eye image and a right eye image of the analysis target frame;
Generating a defocus blur amount (DBA) value for each of the pixels by comparing the blurred image of the frame to be analyzed and the analyzed frame; And
Calculating a visual attention degree for each of the pixels based on the V value, the DFS value, and the DBA value,
Wherein the DFS value is a vertical distance value between a point at which a stereoscopic image is formed by the left eye image and the right eye image and the screen. The method according to claim 1,
Wherein the step of measuring the 3D effect includes weighting a weight of the gradient of the depth image of each pixel of the analysis target frame according to the visual attention degree, . 3. The method of claim 2,
Further comprising the step of calculating a visual attention probability (VAP) having a value of 0 to 1 by normalizing the visual attention degree. The method of claim 3,
The step of calculating the visual attention probability includes:
Equation

Calculating a probability of the visual attention by means of the visual attention probability,
Where i is the longitudinal index of the pixel and j is the lateral index of the pixel. 5. The method of claim 4,
Wherein the step of measuring the three-
Equation

Calculating a weighted gradient magnitude by the VAP (WGMVAP) by the visual attention probability; And
Measuring a 3D sensation on the 3D image file based on the WGMVAP,
The VSK is a visual sensitivity kernel,
remind

Is the magnitude of the gradient of the depth image of the pixel whose coordinates are (x, y)
Wherein i and x are longitudinal indexes of pixels and j and y are lateral indexes of pixels. 6. The method of claim 5,
Wherein the visual sensitivity kernel has a shape that linearly decreases in a horizontal direction and a vertical direction from a point where the time is concentrated. delete The method according to claim 1,
Wherein the visual attention degree for each of the pixels is larger as the V value and the DFS value are larger and the DBA value is smaller. 9. The method of claim 8,
Wherein calculating the visual attention degree for each of the pixels comprises:
And calculating a visual attention degree for each of the pixels by reflecting the interaction between the V value and the DBA value. The method according to claim 1,
The step of generating the DBA value comprises:
Generating a DBA value for each edge pixel included in the analysis target frame by comparing the analysis target frame and the blurred image; And
Generating a DBA value for each of the pixels based on the DBA value for each of the edge pixels included in the analysis target frame and the DFS value. 11. The method of claim 10,
Wherein the step of generating a DBA value for each edge pixel included in the analysis target frame comprises:
Calculating a first gradient that is a gradient of a pixel value of an edge pixel included in the frame to be analyzed;
Generating a blurred image by convoluting the frame to be analyzed and a Gaussian filter;
Calculating a second gradient that is a gradient of pixel values of edge pixels included in the blurred image; And
And generating a DBA value for each edge pixel included in the analysis target frame based on the ratio of the first gradient and the second gradient and the standard deviation of the Gaussian filter. Way. 11. The method of claim 10,
Generating a DBA value for each of all the pixels based on the DBA value for each edge pixel included in the analysis target frame and the DFS value,
Dividing the analysis target frame into a plurality of regions based on the DFS value;
Extracting a reference region that is an area having the smallest average value of the DBA values by calculating an average of DBA values for edge pixels included in each of the plurality of regions; And
And generating the DBA value based on an average of the DBA values of the reference region and a distance between the reference region and each of the pixels. A visual attention degree calculator configured to calculate a visual attention score (VAS) for a frame to be analyzed among a plurality of frames included in the 3D image file; And
And a stereoscopic effect measuring unit configured to measure stereoscopic effect on the 3D image file based on the visual attention degree,
The visual attention degree calculating unit may calculate,
A V value generating unit for generating a velocity (V) value for each of the pixels included in the analysis target frame by comparing the analysis target frame and the previous frame of the analysis target frame among the plurality of frames;
A DFS value generator for generating a distance from the screen (DFS) value for each of the pixels by analyzing a left eye image and a right eye image of the analysis target frame; And
And a DBA value generator for generating a defocus blur amount (DBA) value for each of the pixels by comparing blurred images of the frame to be analyzed and the frame to be analyzed,
Wherein the DFS value is a vertical distance value between a point at which a stereoscopic image is formed by the left eye image and the right eye image and the screen. A visual attention score (VAS) for a frame to be analyzed among a plurality of frames included in a 3D image file is calculated,
And a set of instructions for measuring a stereoscopic effect on the three-dimensional image file based on the visual attention degree,
The calculation of the visual attention degree for the analysis target frame may be performed,
Generating a velocity (V) value for each of the pixels included in the analysis target frame by comparing the analysis target frame and a previous frame of the analysis target frame among the plurality of frames,
Generating a distance from the screen (DFS) value for each of the pixels by analyzing a left eye image and a right eye image of the analysis target frame,
Generating a defocus blur amount (DBA) value for each of the pixels by comparing the blurred image of the frame to be analyzed and the blurred image of the frame to be analyzed,
And calculating a visual attention degree for each of the pixels based on the V value, the DFS value, and the DBA value,
Wherein the DFS value is a vertical distance value between a point at which a stereoscopic image is formed by the left eye image and the right eye image and the screen.

Images