JP2014112794A - Stereoscopic image generating apparatus, stereoscopic image generating method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate stereoscopic image data prevented from causing motion sickness.SOLUTION: A most projecting pixel presence possible region setting unit 11 sets an area which can include a pixel having a largest projecting amount, to inputted image data. A largest projecting amount calculation unit 12 calculates the largest projecting amount of a pixel in the set area. A stereoscopic image generating unit 13 determines the largest projecting amount of the pixel in the stereoscopic image data on the basis of the calculated largest projecting amount, and generates the stereoscopic image data from the image data on the basis of the determined largest projecting amount.

Description

  The present invention relates to a technique for generating stereoscopic image data.

  With the advancement of digital technology, stereoscopic image generating apparatuses capable of capturing and editing stereoscopic images are becoming widespread. For example, it is possible to shoot a stereoscopic image, which conventionally requires a large-scale device, with a hand-held camcorder. With the spread of such 3D image generation devices, the number of 3D images to be disseminated is increasing rapidly.

  Humans have a three-dimensional effect due to parallax, vergence, or focus adjustment of the eyeball. In natural vision, the position of the object recognized by parallax or convergence matches the position of the object recognized by the focus adjustment function of the eyeball, and there is no sense of incongruity. However, when viewing the stereoscopic image displayed on the stereoscopic image display device, the position of the object recognized by the parallax or the convergence is the front direction or the back direction with respect to the display surface of the stereoscopic image display device. The position of the object recognized by the focus adjustment function of the eyeball is on the display surface of the stereoscopic image display device. Therefore, the position information of the object is inconsistent, and the user feels uncomfortable. Further, when the object moves, the difference between the position of the object recognized by parallax or convergence and the position of the object recognized by the focus adjustment function of the eyeball changes. Due to the change in the positional information difference, the above-mentioned uncomfortable feeling increases more and more, and as a result, the user feels more tired or feels headache or nausea. Such a state is called video sickness. That is, it can be said that viewing a stereoscopic image is more likely to cause video sickness than viewing a planar image.

  A stereoscopic image created by a consumer with a household device is not always considered enough to prevent motion sickness. Therefore, it can be said that a stereoscopic image created by a general consumer has a relatively high risk of causing motion sickness. On the other hand, it can be said that a stereoscopic image created by a person with specialized knowledge with sufficient consideration for prevention of video sickness has a relatively low risk of causing video sickness. However, depending on the physical condition and viewing environment of the viewer, there is a possibility of causing video sickness even if the stereoscopic image is created with sufficient consideration for prevention of video sickness.

  Patent Documents 1 and 2 disclose techniques for reducing video sickness caused by stereoscopic images. In other words, the technique disclosed in Patent Document 1 prevents viewers from getting sick of video by replacing and displaying an image with little sudden change when the amount of gaze shift of the viewer exceeds a threshold value. . The technique disclosed in Patent Document 2 adjusts the amount of parallax based on parallax adjustment information and generates a stereoscopic image based on the adjusted amount of parallax, thereby reducing discomfort and preventing video sickness.

JP 2010-266989 A JP 2011-35712 A

  However, since the technique disclosed in Patent Document 1 is a technique for preventing video sickness based on the viewer's gaze shift amount, it is implemented in a stereoscopic image display apparatus that does not have a means for measuring the viewer's gaze shift amount. Can not do it. For the same reason, the technique disclosed in Patent Document 1 cannot be applied to the production of a stereoscopic image in which the occurrence of video sickness is suppressed. Furthermore, the technique disclosed in Patent Document 1 is difficult to deal with when there are a plurality of viewers because the amount of line-of-sight shift may differ for each viewer. In addition, since the technique disclosed in Patent Document 2 does not consider line-of-sight shift, video sickness caused by line-of-sight shift cannot be prevented.

  Accordingly, an object of the present invention is to generate stereoscopic image data in which video sickness is prevented.

  The stereoscopic image generating apparatus of the present invention is configured to set an area for which a pixel having a maximum pop-out amount may exist for the image data input by the input means and the input means for inputting image data. Means for calculating the maximum pop-out amount of the pixel in the region set by the setting means, and based on the maximum pop-out amount calculated by the calculation means, the maximum pixel amount in the stereoscopic image data It has a determining means for determining the pop-out amount and a generating means for generating the stereoscopic image data from the image data based on the maximum pop-out amount determined by the determining means.

  According to the present invention, it is possible to generate stereoscopic image data in which video sickness is prevented.

It is a figure which shows the structure of the stereo image production | generation apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the detailed structure of the largest protrusion pixel presence possible area | region setting part in the 1st Embodiment of this invention. It is a figure which shows the detailed structure of a movement destination coordinate estimation part. It is a flowchart which shows the process of the largest protrusion pixel presence possible area | region selection part. It is a flowchart which shows the determination process of the parallax amount of the stereo image data by a stereo image production | generation part. It is a figure which shows an example of the stereo image data in a comparative example, and the position of the largest pop-out pixel. It is a figure which shows an example of the stereo image data in a comparative example, and the position of the largest pop-out pixel. It is a figure which shows an example of the stereo image data produced | generated by the stereo image production | generation apparatus which concerns on embodiment of this invention, and the position of the largest pop-out pixel. It is a figure which shows an example of the stereo image data produced | generated by the stereo image production | generation apparatus which concerns on this embodiment, and the position of the largest pop-out pixel. It is a figure for demonstrating the transition of the position of the largest protrusion pixel from the time n to the time n + 6. It is a figure which shows the structure of the stereo image production | generation apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the detailed structure of the largest protruding pixel presence possible area | region setting part in the 2nd Embodiment of this invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings. However, the configuration can be changed as appropriate without departing from the gist of the present invention. Therefore, the scope of application of the present invention is not limited to the embodiments described below. In the following description, components having the same functions are denoted by the same reference numerals in principle, and redundant descriptions are omitted.

  Prior to the description of the embodiment of the present invention, the relationship between the imaging positions of a stereoscopic image will be described. When a stereoscopic image is viewed with an eyeball, the viewer recognizes that the images are fused in the brain due to parallax and convergence of both eyes, and the image is projected from the display surface. At this time, the pop-out amount is larger as the object is closer to the viewer and smaller as the object is farther from the viewer.

  FIG. 1 is a diagram illustrating a configuration of a stereoscopic image generation apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the stereoscopic image generation apparatus according to the present embodiment includes a maximum pop-out pixel presence area setting unit 11, a maximum pop-out amount calculation unit 12, and a three-dimensional image generation unit 13. Note that the maximum pop-out pixel existence area setting unit 11, the maximum pop-out amount calculation unit 12, and the stereoscopic image generation unit 13 are such that a CPU (not shown) in the stereoscopic image generation apparatus downloads necessary data and programs from a recording medium such as ROM. This is a functional configuration realized by reading and executing. As another embodiment, at least a part of the maximum pop-out pixel presence area setting unit 11, the maximum pop-out amount calculation unit 12, and the stereoscopic image generation unit 13 may be implemented by hardware.

  The maximum pop-out pixel existence area setting unit 11 calculates and outputs a coordinate value 102 indicating an area where a pixel having the maximum pop-out amount may exist at the next time. In the following, “pixels with the largest pop-out amount” will be referred to as “maximum pop-out pixels”, and “regions where pixels with the largest pop-out amount may exist” are referred to as “maximum pop-out pixel existence regions”. Called.

  FIG. 2 is a diagram illustrating a detailed configuration of the maximum pop-out pixel existence area setting unit 11. As shown in FIG. 2, the maximum pop-out pixel existence region setting unit 11 includes a movement destination coordinate detection unit 21, a movement destination coordinate prediction unit 22, a scene change determination unit 23, a maximum pop-out pixel existence region selection unit 24, and A delay unit 25 is provided.

  The destination coordinate detection unit 21 detects the coordinate value at which the maximum pop-out pixel at the current time exists at the next time, that is, the destination coordinate. In the present embodiment, an image correlation method is used in which the pixel movement amount and the movement destination coordinates are detected from the correlation between the image data at the current time and the image data at the next time. However, the present invention is not limited to this. The current time is an example of the first time point, and the next time is an example of the second time point. The image data at the current time is an example of first image data, and the image data at the next time is an example of second image data.

  The delay unit 25 delays the input image data 101 at the current time by one unit time in order to match the time with the input image data 101 at the next time. The destination coordinate detection unit 21 correlates the input image data 111 at the current time delayed by the delay unit 25 with the input image data 101 at the next time, and the destination coordinate 112 and the presence / absence 113 of the destination coordinate exist. Is detected. Here, in order to calculate the coordinate value 102 of the maximum pop-out pixel existence possible area, only the movement destination coordinate of the maximum pop-out pixel at the current time and its presence / absence are required, and other pixels are unnecessary. Accordingly, the movement destination coordinate detection unit 21 detects the movement destination coordinates 112 and the presence / absence of the movement destination coordinates 113 only for the pixel indicated by the coordinate value 103 of the maximum protruding pixel at the current time. The presence / absence 113 of the movement destination coordinate takes a value of TRUE when the movement destination coordinate 112 exists, and takes a value of FALSE when the movement destination coordinate 112 does not exist.

  The destination coordinate prediction unit 22 predicts a destination coordinate where the maximum pop-out pixel at the current time will exist at the next time, and outputs it as a predicted coordinate value 114. In the present embodiment, the movement destination coordinates are predicted using the motion vector, but the present invention is not limited to this.

  FIG. 3 is a diagram illustrating a detailed configuration of the destination coordinate prediction unit 22. As illustrated in FIG. 3, the movement destination coordinate prediction unit 22 includes a motion vector calculation unit 26, a movement destination coordinate calculation unit 28, and a delay unit 25.

  The motion vector calculation unit 26 calculates a motion vector 116 from the input image data 101 at the next time and the input image data 111 at the current time after the delay. The delay unit 25 generates the delayed motion vector 116 by matching the time of the motion vector 116 with the coordinate value 103 of the maximum pop-out pixel at the current time. The destination coordinate calculation unit 28 calculates a predicted coordinate value 114 by performing a vector calculation on the delayed motion vector 117 and the coordinate value 103 of the maximum pop-out pixel at the current time. Note that the method of calculating the predicted coordinate value 114 is not limited to the method using the motion vector described above.

The scene change determination unit 23 determines the presence / absence of a scene change between the current time and the next time, and outputs a scene change flag 115. When there is a scene change between the current time and the next time, the scene change flag 115 takes a value of TRUE, whereas when there is no scene change, the scene change flag 115 takes a value of FALSE. As a scene change determination method, for example, a method using a motion vector disclosed in the following document can be applied.
<Reference>
Sonehara et al .: Scene change detection using motion vectors, Proceedings of the Annual Conference of the Institute of Image Information and Television Engineers, 1997-07-29, pp 304 (1997)

  The maximum pop-out pixel existence area selection unit 24 generates the coordinate value 102 of the maximum pop-out pixel existence area based on the movement destination coordinates 112, the presence / absence of movement destination coordinates 113, and the scene change flag 115 according to the following policy. . Here, Px is the X component of the predicted coordinate value 114, and Py is the Y component of the predicted coordinate value 114. In addition, the horizontal size of the display surface is Dx, the vertical size is Dy, and the upper left pixel of the display surface is the origin (0, 0). Accordingly, the range of the pixel coordinates (Ix, Iy) is {Ix: 0 <= Ix <= Dx-1} and {Iy: 0 <= Iy <= Dy-1}.

・ Policy 1
When the maximum pop-out pixel at the current time exists at the next time (presence / absence of movement destination coordinates 113 == TRUE), the maximum pop-out pixel existence area selection unit 24 determines that the center of the maximum protrusion pixel existence area is the movement destination coordinates 112 Then, the coordinate value 102 of the maximum pop-out pixel existence possible area is calculated.

・ Policy 2
When the maximum pop-out pixel at the current time does not exist at the next time and there is a scene change (presence / absence of movement destination coordinates 113 == TRUE AND scene change flag 115 == TRUE), the maximum pop-out pixel existence area selection unit In step 24, the maximum pop-out pixel existence region is determined for the entire display screen.

・ Policy 3
When the maximum pop-out pixel at the current time does not exist at the next time, there is no scene change, and the predicted coordinate value 114 is within the display screen {presence / absence of movement destination coordinate 113 == FALSE AND scene change flag 115 == FALSE AND (0 <= Px <= Dx-1) AND (0 <= Py <= Dy-1)}, the maximum pop-out pixel existence area selection unit 24 has a predicted coordinate value at the center of the maximum pop-out pixel existence area. The coordinate value 102 of the maximum pop-out pixel existence possible area is calculated so as to be 114.

・ Policy 4
When the maximum pop-out pixel at the current time does not exist at the next time, there is no scene change, and the predicted coordinate value 114 is outside the display screen {presence / absence of destination coordinate 113 == FALSE AND scene change flag 115 == FALSE AND (0> Px OR Dx <Px) OR (0> Py OR Dy <Py)}, the maximum pop-out pixel existence area selection unit 24 sets the center of the maximum pop-out pixel existence area to the maximum pop-out pixel at the current time. The coordinate value 102 of the maximum pop-out pixel existing area is calculated so that the coordinate value 103 is obtained.

  Except when the maximum pop-out pixel at the current time does not exist at the next time and there is a scene change, the shape and size of the maximum pop-out pixel existence area are determined depending on the size and characteristics of the display screen. . For example, the shape and size of the maximum pop-out pixel existence area may be determined using the viewing angle.

  FIG. 4 is a flowchart showing processing of the maximum pop-out pixel existence possible area selection unit 24. In step S <b> 101, the maximum pop-out pixel existence area selection unit 24 determines whether or not the maximum pop-out pixel at the current time exists at the next time. If the maximum pop-out pixel exists at the next time, the process proceeds to step S102. On the other hand, if the maximum pop-out pixel does not exist at the next time, the process proceeds to step S103.

  In step S102, the maximum pop-out pixel existence region selection unit 24 calculates the coordinate value 102 of the maximum pop-out pixel existence region so that the center of the maximum pop-out pixel existence region becomes the movement destination coordinate 112 (policy 1). . In step S103, the maximum pop-out pixel existence possible area selection unit 24 determines whether or not there is a scene change. If there is a scene change, the process proceeds to step S104. On the other hand, if there is no scene change, the process proceeds to step S105.

  In step S104, the maximum pop-out pixel existence area selection unit 24 determines the maximum pop-out pixel existence area on the entire display screen (policy 2). In step S105, the maximum pop-out pixel existence possible area selection unit 24 determines whether or not the predicted coordinate value 114 is within the display screen. If the predicted coordinate value 114 is within the display screen, the process proceeds to step S106. On the other hand, if the predicted coordinate value 114 is not within the display screen, the process proceeds to step S107.

  In step S106, the maximum pop-out pixel existence area selection unit 24 calculates the coordinate value 102 of the maximum pop-out pixel existence area so that the center of the maximum pop-out pixel existence area becomes the predicted coordinate value 114 (policy 3). . In step S107, the maximum pop-out pixel existence area selection unit 24 sets the coordinate value 102 of the maximum pop-out pixel existence area so that the center of the maximum pop-out pixel existence area becomes the coordinate value 103 of the maximum pop-out pixel at the current time. Calculate (policy 4).

  Next, the processing of the maximum pop-out amount calculation unit 12 will be described. The maximum pop-out amount calculation unit 12 calculates the maximum pop-out pixel coordinate value 103 and the maximum pop-out pixel pop-out amount 104 within the range of the maximum pop-out pixel existence area with respect to the input image data 101. Since the pop-out amount is proportional to the parallax, the maximum pop-out amount calculation unit 12 sets the coordinate value of the pixel having the largest parallax within the range of the maximum pop-out pixel existence area as the coordinate value 103 of the maximum pop-out pixel, and sets the parallax amount. The maximum pop-out pixel pop-out amount 104 is determined. When there are a plurality of pixels having the largest parallax within the range of the maximum pop-out pixel existence area, the maximum pop-out amount calculation unit 12 determines the coordinate value of the pixel closest to the center of the maximum pop-out pixel existence area as the maximum pop-out pixel existence area. The coordinate value 103 is determined. The maximum pop-out amount calculation unit 12 determines the parallax amount as the pop-out amount 104 of the maximum pop-out pixel. Further, when there are a plurality of pixels having the largest parallax at the closest distance from the center of the maximum pop-out pixel existence area, the maximum pop-out amount calculation unit 12 selects the coordinate value of the pixel arbitrarily selected from the corresponding pixels. Is determined as the coordinate value 103 of the maximum pop-out pixel. Then, the maximum pop-out amount calculation unit 12 determines the parallax amount as the pop-out amount 104 of the maximum pop-out pixel.

  Next, processing of the stereoscopic image generation unit 13 will be described. The stereoscopic image generation unit 13 generates stereoscopic image data 105 from the input image data 101. In generating the stereoscopic image data 105, the pop-out amount of the stereo image data 105, that is, the parallax amount of the stereo image data 105 is determined so as not to exceed the pop-out amount 104 of the maximum pop-out pixel.

  FIG. 5 is a flowchart illustrating the determination process of the parallax amount of the stereoscopic image data 105 by the stereoscopic image generation unit 13. In step S <b> 201, the stereoscopic image generation unit 13 causes the amount of parallax included in a certain pixel of the input image data 101 or the amount of parallax added to the input image data 101 to exceed the pop-out amount 104 of the maximum pop-out pixel. It is determined whether or not. If the maximum pop-out pixel pop-out amount 104 is exceeded, the process proceeds to step S202. On the other hand, when the maximum pop-out pixel pop-out amount 104 is not exceeded, that is, the amount of parallax that a certain pixel of the input image data 101 has or the amount of parallax added to the input image data 101 is the maximum pop-out pixel. If the pop-out amount is 104 or less, the process proceeds to step S203.

  In step S <b> 202, the stereoscopic image generation unit 13 determines the parallax amount of the stereoscopic image data 105 while suppressing the amount of protrusion 104 of the maximum protruding pixel. In step S <b> 203, the stereoscopic image generation unit 13 determines the amount of parallax of the stereoscopic image data 105 as the amount of parallax included in the pixels of the input image data 101 or the amount of parallax added to the input image data 101.

  6 and 7 are diagrams illustrating an example of the stereoscopic image data and the position of the maximum protruding pixel in the comparative example. 8 and 9 are diagrams showing an example of the stereoscopic image data generated by the stereoscopic image generating apparatus according to the present embodiment and the position of the maximum pop-out pixel. Hereinafter, the difference in the position of the maximum pop-out pixel given in the comparative example and the present embodiment will be described with reference to FIGS.

  First, the relationship between the stereoscopic image data and the position of the largest pop-out pixel in the comparative example will be described with reference to FIGS. FIG. 6 shows a state from time n to time n + 3, and FIG. 7 shows a state from time n + 3 to time n + 6.

(Time n)
FIG. 6A shows stereoscopic image data at time n. Only the person A41 is shown in the stereoscopic image data at time n shown in FIG. In the stereoscopic image data at time n, the nose of the person A41 is present in the foreground, so that the maximum pop-out pixel 121 corresponds to the nose of the person A41 as shown in FIG.

(Time n + 1)
FIG. 6B shows stereoscopic image data at time n + 1. In the stereoscopic image data at time n + 1 shown in FIG. 6B, a person B42 appears in front of the person A41. In the stereoscopic image data at the time n + 1, since the nose of the person B42 is present in the foreground, the maximum pop-out pixel 121 corresponds to the nose of the person B42 as shown in FIG.

(Time n + 2)
FIG. 6C shows stereoscopic image data at time n + 2. As shown in FIG. 6C, in the stereoscopic image data at time n + 2, the person B42 disappears from the display screen, and the person A41 comes out to the front. In the stereoscopic image data at the time n + 2, the nose of the person A41 is present in the foreground, and therefore the maximum pop-out pixel 121 corresponds to the nose of the person A41 as shown in FIG. However, since the person A41 has moved forward, the position of the maximum pop-out pixel 121 is different from the time n.

(Time n + 3)
FIG. 6D and FIG. 7A show the stereoscopic image data at time n + 3. In the stereoscopic image data at time n + 3 shown in FIGS. 6D and 7A, the ball A43 appears on the right side of the display screen and in front of the person A41. In the three-dimensional image data at time n + 3, the ball A43 is closest to the front, so that the maximum protruding pixel 121 corresponds to the ball A43, as shown in FIGS. 6 (h) and 7 (e).

(Time n + 4)
FIG. 7B shows stereoscopic image data at time n + 4. In the stereoscopic image data at time n + 4 shown in FIG. 7B, the ball A43 is reflected in front of the display screen of the person A41, and the ball B44 is reflected on the right side of the display screen. Since the ball B44 exists in front of the ball A43, and the ball B44 exists in the foremost side, the maximum protruding pixel 121 corresponds to the ball B44 as shown in FIG.

(Time n + 5)
FIG. 7C shows stereoscopic image data at time n + 5. In the stereoscopic image data at time n + 5 shown in FIG. 7C, the ball A43 disappears from the display screen, and the person A41 and the ball B44 are shown. Since the ball B44 is present before the time n + 4 and the ball B44 is present at the foremost position, the maximum protruding pixel 121 corresponds to the ball B44 as shown in FIG. However, since the ball B44 has moved forward, the position of the maximum pop-out pixel 121 is different from the time n + 4.

(Time n + 6)
FIG. 7D shows stereoscopic image data at time n + 6. As shown in FIGS. 7C and 7D, there is a scene change between time n + 5 and time n + 6, and the images at time n + 5 and time n + 6 are different. In the stereoscopic image data at time n + 6, since the automobile 45 is present in the foreground, the maximum pop-out pixel 121 corresponds to the left front of the automobile as shown in FIG.

  Next, the relationship between the stereoscopic image data and the position of the largest pop-out pixel in this embodiment will be described with reference to FIGS. FIG. 8 shows a state from time n to time n + 3, and FIG. 9 shows a state from time n + 3 to time n + 6. In the present embodiment, the shape of the maximum pop-out pixel existence possible area 51 is an ellipse. Of course, the shape of the maximum protruding pixel existence region 51 is not limited to an ellipse.

(Time n)
FIG. 8A shows stereoscopic image data at time n. Only the person A41 is shown in the stereoscopic image data at time n shown in FIG. In the three-dimensional image data at time n, since the nose of the person A41 is present in the foreground, the maximum pop-out pixel 121 corresponds to the nose of the person A41 as shown in FIG.

(Time n + 1)
FIG. 8B shows stereoscopic image data at time n + 1. In the stereoscopic image data at time n + 1 shown in FIG. 8B, the person B42 appears in front of the person A41. The nose of the person A41, which is the maximum pop-out pixel 121 shown in FIG. 8E, also exists in the stereoscopic image data at time n + 1. Therefore, the value of presence / absence 113 of the movement destination coordinates is TRUE. Therefore, the maximum pop-out pixel existence area setting unit 11 sets the maximum pop-out pixel existence area 51 around the movement destination coordinates 112 of the maximum pop-out pixel according to the policy 1. Then, the maximum pop-out amount calculation unit 12 determines the pixel having the maximum pop-out amount within the maximum pop-out pixel existence area 51 as the maximum pop-out pixel 121.

  As shown in FIG. 8F, the maximum protruding pixel 121 corresponds to the nose of the person A41. The stereoscopic image generation unit 13 generates stereoscopic image data in which the pop-out amount is suppressed so as to be equal to or less than the pop-out amount of the maximum pop-out pixel 121. Accordingly, as shown in FIG. 7F, the pop-out amount of the person B42 is suppressed as compared with the case where the maximum pop-out pixel 121 is given to the nose of the person B42.

(Time n + 2)
FIG. 8C shows stereoscopic image data at time n + 2. In the stereoscopic image data at time n + 2 shown in FIG. 8C, the person B42 disappears from the display screen, and the person A41 comes out to the front. The maximum protruding pixel 121 at time n + 1, that is, the nose of the person A41 also exists in the stereoscopic image data at time n + 2. Therefore, the value of presence / absence 113 of the movement destination coordinates is TRUE. Therefore, the maximum pop-out pixel existence area setting unit 11 sets the maximum pop-out pixel existence area 51 around the movement destination coordinates 112 of the maximum pop-out pixel according to the policy 1. Then, the maximum pop-out amount calculation unit 12 determines the pixel having the maximum pop-out amount within the maximum pop-out pixel existence area 51 as the maximum pop-out pixel 121.

  As shown in FIG. 8G, the maximum protruding pixel 121 corresponds to the nose of the person A41. However, since the person A41 has moved forward, the position of the maximum pop-out pixel 121 is different from the time n + 1.

(Time n + 3)
FIG. 8D and FIG. 9A show stereoscopic image data at time n + 3. In the stereoscopic image data at time n + 3 shown in FIGS. 8D and 9A, the ball A43 appears on the right side of the display screen and in front of the person A41. The maximum protruding pixel 121 at time n + 2, that is, the nose of the person A41 is present on the stereoscopic image data at time n + 3. Therefore, the value of presence / absence 113 of the movement destination coordinates is TRUE. Therefore, the maximum pop-out pixel existence area setting unit 11 sets the maximum pop-out pixel existence area 51 around the movement destination coordinates 112 of the maximum pop-out pixel according to the policy 1. Then, the maximum pop-out amount calculation unit 12 determines the pixel having the maximum pop-out amount within the maximum pop-out pixel existence area 51 as the maximum pop-out pixel 121.

  As shown in FIG. 8H, the maximum protruding pixel 121 corresponds to the nose of the person A41. The stereoscopic image generation unit 13 generates stereoscopic image data in which the pop-out amount is suppressed so as to be equal to or less than the pop-out amount of the maximum pop-out pixel 121. As a result, as shown in FIG. 6F, the pop-out amount of the ball A43 is suppressed as compared with the case where the maximum pop-out pixel 121 is given to the ball A43.

(Time n + 4)
FIG. 9B shows stereoscopic image data at time n + 4. In the stereoscopic image data at time n + 4 shown in FIG. 9B, the ball A43 is reflected in front of the person A41 and the ball B44 is reflected on the right side of the display screen. The maximum protruding pixel 121 at time n + 3, that is, the nose of the person A41 does not exist behind the ball A43 on the stereoscopic image data at time n + 4. Therefore, the value of presence / absence 113 of the movement destination coordinates is FALSE. Further, the predicted coordinate value 114 of the largest pop-out pixel exists in the display screen. Therefore, the maximum pop-out pixel existence region setting unit 11 sets the maximum pop-out pixel existence region 51 around the predicted coordinate value 114 of the maximum pop-out pixel according to the policy 3. Then, the maximum pop-out amount calculation unit 12 determines the pixel having the maximum pop-out amount within the maximum pop-out pixel existence area 51 as the maximum pop-out pixel 121.

  As shown in FIG. 9F, the maximum protruding pixel 121 corresponds to the ball A43. The stereoscopic image generation unit 13 generates stereoscopic image data in which the pop-out amount is suppressed so as to be equal to or less than the pop-out amount of the maximum pop-out pixel 121. As a result, as shown in FIG. 7F, the amount of protrusion of the ball B44 is suppressed as compared with the case where the maximum protrusion pixel 121 is given to the ball B44.

(Time n + 5)
FIG. 9C shows stereoscopic image data at time n + 5. In the stereoscopic image data at time n + 5 shown in FIG. 9C, the ball A43 disappears from the display screen, and the person A41 and the ball B44 are shown. At time n + 5, the ball B44 is present before the time n + 4. The maximum pop-out pixel 121 at time n + 4, that is, the ball A43, has left the display screen at time n + 5 and does not exist on the display screen. Therefore, the value of presence / absence 113 of the movement destination coordinate is FALSE, and the predicted coordinate value 114 of the maximum pop-out pixel is also outside the display screen. Therefore, the maximum pop-out pixel existence region setting unit 11 sets the maximum pop-out pixel existence region 51 around the coordinates of the maximum pop-out pixel 121 at time n + 4 according to the policy 4. Then, the maximum pop-out amount calculation unit 12 determines the pixel having the maximum pop-out amount within the maximum pop-out pixel existence area 51 as the maximum pop-out pixel 121.

  As shown in FIG. 9G, the maximum pop-out pixel 121 corresponds to the nose of the person A41. The stereoscopic image generation unit 13 generates stereoscopic image data in which the pop-out amount is suppressed so as to be equal to or less than the pop-out amount of the maximum pop-out pixel 121. As a result, as shown in FIG. 7G, the pop-out amount of the ball B44 is suppressed as compared with the case where the maximum pop-out pixel 121 is given to the ball B44.

(Time n + 6)
FIG. 7D shows stereoscopic image data at time n + 6. Since there is a scene change between the time n + 5 and the time n + 6, the video is different at the time n + 5 and the time n + 6. Therefore, the maximum pop-out pixel existence area setting unit 11 determines the pixel having the maximum pop-out amount on the entire display screen as the maximum pop-out pixel 121 according to the policy 2. In the stereoscopic image data at time n + 6, since the automobile 45 is present in the foreground, the maximum pop-out pixel 121 corresponds to the left front of the automobile as shown in FIG.

  Next, transition of the position of the maximum pop-out pixel 121 from time n to time n + 6 will be described with reference to FIG. FIG. 10A shows a path along which the maximum protruding pixel 121 changes in the comparative example. FIG. 10B shows a path along which the maximum pop-out pixel 121 changes in the present embodiment. In FIG. 10, numbers in parentheses indicate the order of transition of the maximum pop-out pixel 121.

  As shown in FIG. 10A, the transition of the maximum protruding pixel 121 in the comparative example is large, and the transition direction is various. On the other hand, as shown in FIG. 10B, the transition of the maximum protruding pixel 121 in this embodiment is small. Further, the transition direction is along the movement direction of the person A41.

  Since the viewer pays attention to the region where the pop-out amount is large in the stereoscopic image data, the shift of the line of sight increases when the shift of the maximum pop-out pixel 121 is large. In addition, when the line of sight changes greatly, the accumulation of fatigue is large, and video sickness is likely to occur. In addition, when the line-of-sight shift direction is irregular, video sickness is likely to occur.

  Therefore, according to the present embodiment, it is possible to suppress the shift of the maximum pop-out pixel 121 and to suppress the shift amount of the line of sight accompanying viewing of the stereoscopic image data. Further, according to the present embodiment, by restricting the transition direction of the maximum pop-out pixel 121, it is possible to prevent an irregular line-of-sight transition that is a major cause of video sickness.

  Next, a second embodiment of the present invention will be described. FIG. 11 is a diagram illustrating a configuration of a stereoscopic image generating apparatus according to the second embodiment of the present invention. The difference between the stereoscopic image generating apparatus according to the second embodiment shown in FIG. 11 and the stereoscopic image generating apparatus according to the first embodiment shown in FIG. 1 is that the maximum pop-out pixel existence region setting in the second embodiment is performed. The unit 1111 has a function of changing the shape and size of the maximum pop-out pixel existence area in addition to the function of setting the maximum pop-out pixel existence area described in the first embodiment. In other words, the maximum pop-out pixel existence area setting unit 1111 in the second embodiment changes the shape and size of the maximum pop-out pixel existence area according to the value of the input maximum pop-out pixel existence area attribute 106.

  The maximum pop-out pixel existence region attribute 106 is an attribute value representing the shape and size of the maximum pop-out pixel existence region. Note that the maximum pop-out pixel existence region attribute is a value that can be arbitrarily set by the user of the stereoscopic video generation apparatus according to the present embodiment. As a method for setting the maximum pop-out pixel existence region attribute, for example, there is a method in which the user sets an arbitrary maximum pop-out pixel existence region attribute using a remote controller or OSD (On-Screen-Display).

  FIG. 12 is a diagram illustrating a detailed configuration of the maximum pop-out pixel existence area setting unit 1111 in the second embodiment. The difference between the maximum pop-out pixel existence region setting unit 1111 in the second embodiment shown in FIG. 11 and the maximum pop-out pixel existence region setting unit 11 in the first embodiment shown in FIG. This is in the function of the area selection unit 1124.

  That is, the maximum pop-out pixel existence area selection unit 1124 according to the second embodiment determines the position of the maximum pop-out pixel existence area according to the value of the maximum pop-out pixel existence area attribute 106, and determines the maximum pop-out pixel existence area. A coordinate value 102 is calculated.

Here, a specific example of the maximum pop-out pixel existence region attribute 106 will be described. The maximum pop-out pixel existence area attribute 106 takes a 4-bit value, and the shape is represented by the upper 2 bits and the size is represented by the lower 2 bits. For example, the relationship between the upper 2 bits and the shape and the relationship between the lower 2 bits and the size are as follows.
Upper 2 bits of the maximum pop-out pixel existence area attribute 106 00 Not used 01 Circle
10 rounded rectangle
11 Ellipse
Lower 2 bits of maximum pop-out pixel existence area attribute 106 00 Not used 01 Small (horizontal length 5 cm on display screen)
10 Medium (Length 10cm in the horizontal direction on the display screen)
11 large (horizontal length of 15 cm on the display screen)

  Note that the ratio of the horizontal size and the vertical size of rounded squares and ellipses is, for example, horizontal: vertical = 1.2: 1.0. For example, when the pixel existence possibility area attribute 106 = 1110, the maximum pop-out pixel existence possibility area is an ellipse having a horizontal length of 10 cm on the display screen. Of course, the form of the pixel existence possible area attribute 106 and the shape and size of the pixel existence possible area are not limited to the above-described example.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  11, 1111: Maximum pop-out pixel existence possible area setting unit, 12: Maximum pop-out amount calculation unit, 13: Stereo image generation unit, 21: Destination coordinate detection unit, 22: Destination coordinate prediction unit, 23: Scene change determination unit , 24, 1124: maximum pop-out pixel existence region selection unit, 25: delay unit, 26: motion vector calculation unit, 28: destination coordinate calculation unit

Claims (8)

Input means for inputting image data;
Setting means for setting a region where a pixel having the maximum pop-out amount may exist for the image data input by the input means;
Calculating means for calculating the maximum amount of popping out of the pixels in the region set by the setting means;
Determining means for determining the maximum pop-out amount of the pixels in the stereoscopic image data based on the maximum pop-out amount calculated by the calculating means;
A stereoscopic image generating apparatus comprising: generating means for generating the stereoscopic image data from the image data based on the maximum pop-out amount determined by the determining means.   2. The stereoscopic image generation according to claim 1, wherein the determining unit determines a maximum pop-out amount of a pixel in the stereoscopic image data so as not to exceed a maximum pop-out amount calculated by the calculating unit. apparatus.   In the setting unit, a pixel having the maximum pop-out amount in the first image data input at the first time point by the input unit is input by the input unit at a second time point after the first time point. 3. The stereoscopic image generation apparatus according to claim 1, wherein the region is set based on a position of the pixel in the second image data when the second image data exists.   The setting means predicts a movement destination of the pixel at the second time point when a pixel with the maximum pop-out amount in the first image data does not exist in the second image data, and The stereoscopic image generating apparatus according to claim 3, wherein the area is set based on a destination.   The three-dimensional object according to claim 3 or 4, wherein the setting means sets the entire display screen as the region when a scene change exists between the first time point and the second time point. Image generation device.   The stereoscopic image generation according to any one of claims 1 to 5, further comprising changing means for changing at least one of the shape and size of the area set by the setting means. apparatus. A stereoscopic image generation method executed by a stereoscopic image generation apparatus,
An input step for inputting image data;
A setting step for setting an area where there is a possibility that a pixel having a maximum pop-out amount exists for the image data input in the input step;
A calculation step for calculating a maximum amount of protrusion of the pixel in the region set by the setting step;
A determination step for determining a maximum amount of pop-up of pixels in the stereoscopic image data based on the maximum amount of pop-up calculated by the calculating step;
And a generating step of generating the stereoscopic image data from the image data based on the maximum pop-out amount determined in the determining step. An input step for inputting image data;
A setting step for setting an area where there is a possibility that a pixel having a maximum pop-out amount exists for the image data input in the input step;
A calculation step for calculating a maximum amount of protrusion of the pixel in the region set by the setting step;
A determination step for determining a maximum amount of pop-up of pixels in the stereoscopic image data based on the maximum amount of pop-up calculated by the calculating step;
A program for causing a computer to execute a generation step of generating the stereoscopic image data from the image data based on the maximum pop-out amount determined by the determination step.

Images