JP2012034000A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve image quality of a three-dimensional (3D) image when adding grain noise to the 3D image.SOLUTION: An image processing apparatus for processing a three-dimensional (3D) image includes plural random noise generating units. The random noise generating units generate n numbers (n>2) of random noise which do not have correlation with one another, and by adding up n-1 numbers of random noise, including m numbers (n-1>m>0) of arbitrary random noise from the generated random noise in common, generate two pieces of grain noise for an L image and an R image which have a prescribed correlation with one another. The grain noise for the L image is added to the L image, and the grain noise for the R image is added to the R image. The L image and the R image having the grain noise added thereto are alternately displayed. The present invention can be applied to a television receiver which performs processing on a 3D image.

Description

  The present invention relates to an image processing device, an image processing method, and a program, and more particularly to an image processing device, an image processing method, and a program that can improve the image quality of a 3D image when grain noise is added.

  A so-called film grain process is performed to intentionally produce the grain feeling (film grain) of the photosensitive material seen in the material photographed with a film camera by adding a single noise based on random noise to the image. Yes. Film grain processing can enhance the fineness and gradation of an image.

JP 2008-11202 A

  By the way, 3D content including 3D images that can be stereoscopically viewed as video data has attracted attention. As a 3D image display method, there is a frame sequential method in which a left-eye image (L image) and a right-eye image (R image) having parallax between the same objects are alternately displayed. Delivering a sense of depth to the user by delivering the L image and the R image to the left and right eyes of the user wearing 3D glasses such as shutter glasses.

  Intentionally adding grain noise, which is noise equivalent to film grain, is basically considered to be effective for a 3D image including an L image and an R image. There are various methods for adding grain noise to a 3D image.

  FIG. 1 is a diagram illustrating a method of adding grain noise.

  The method of FIG. 1 is a method of adding random noise generated by one random noise generator to each of the L image and the R image.

  As shown in FIG. 2, the same random noise A is added to the image L1 that is the L image and the image R1 that is the R image, respectively, and the image L1 ′ that is the image L1 and the image R1 that is the image R1 to which the random noise A is added. Assume that 'is displayed alternately as indicated by the tip of arrow # 1. There is a shift corresponding to parallax between the position of the object in the image L1 and the position of the object in the image R1. In the example of FIG. 2, a person is shown in the images L1 and R1.

  In this case, since the same noise is added to the same position in the image L1 and the image R1, the parallax between the noise added to the image L1 and the noise added to the image R1 is zero. Therefore, a user wearing 3D glasses is localized on the display surface by planar grain noise (2D grain noise), and sees an image separated from a stereoscopically visible object. In the image ahead of the arrow # 1, the position of the person being shifted indicates that it is possible to see the person stereoscopically by parallax.

  As described above, the method of adding the same random noise to the L image and the R image cannot sufficiently obtain the effect of enhancing the fineness and gradation of the image as obtained in the 2D image.

  FIG. 3 is a diagram illustrating another method for adding grain noise.

  In the method of FIG. 3, random noise is generated in two random noise generation units, random noise generated by one random noise generation unit is added to the L image, and random noise generated by the other random noise generation unit is R This is a method of adding to an image.

  Assuming a situation in which the L image and the R image are taken with different films (cameras), there is no correlation between the arrangement of the photosensitive material of the L image film and the photosensitive material of the R image film. As shown in FIG. 3, if different random noise generators are prepared for the L image and the R image, and random noise is added, the effect of grain noise should be obtained according to the principle.

  Consider a case where random noise A is added to the image L1 and random noise B is added to the image R1 as shown in FIG. 4, and the images L1 ′ and R1 ′ are alternately displayed as indicated by the tip of the arrow # 2. .

  In this case, noise added to the same position in the image L1 and the image R1 is uncorrelated. Therefore, a user wearing 3D glasses will see an image with random noise and bright spots in the space before and after the display surface (the space on the user side and the space on the back side with respect to the display surface). . Since the noise added to the image L1 and the image R1 varies, the noise and the bright spot in the front-rear direction are distributed over a wide range such that the position appears to be extremely deep or the front. Or noise or bright spots appear to move.

  As described above, in the method of adding two uncorrelated random noises to the L image and the R image, respectively, it is possible to sufficiently obtain the effect of enhancing the sense of fineness and gradation of the image as obtained in the 2D image. In addition to not being able to do so, there is a possibility that the user may feel uncomfortable.

  The present invention has been made in view of such circumstances, and is intended to improve the image quality of a 3D image when grain noise is added.

  An image processing apparatus according to an aspect of the present invention generates a first noise composed of noise for each pixel of an image, and a second noise composed of noise for each pixel of the image correlated with the first noise. And generating means for adding the first noise to the image for the left eye and adding the second noise to the image for the right eye.

  The generating means generates n (n> 2) uncorrelated noise, and a sum of arbitrary m (n-1> m> 0) noises out of the n uncorrelated noises. The first noise and the second noise having a correlation of m / (n−1) that include the components represented by the following can be generated.

  Display control means for alternately displaying the left-eye image to which the first noise is added and the right-eye image to which the second noise is added can be further provided.

  The adding means sets parallax to the first noise and the second noise, and the first noise and the second noise having parallax with each other are added to the left-eye image and the right-eye image. Each can be added.

  Detection means for detecting an object included in the left-eye image and the right-eye image can be further provided. In this case, the adding means adds noise added to each pixel constituting the object area detected from the left-eye image and each pixel constituting the object area detected from the right-eye image. At least one of the noises to be added can have the same amount of parallax as the parallax set for the object, and the noise set with the parallax can be added to each pixel constituting the region of the object .

  An image processing method according to an aspect of the present invention generates a first noise composed of noise for each pixel of an image and a second noise composed of noise for each pixel of the image correlated with the first noise. And adding the first noise to the left-eye image and adding the second noise to the right-eye image.

  The program according to one aspect of the present invention generates a first noise composed of noise for each pixel of an image and a second noise composed of noise for each pixel of the image correlated with the first noise, A computer is caused to execute a process including a step of adding the first noise to the left-eye image and adding the second noise to the right-eye image.

  In one aspect of the present invention, a first noise composed of noise for each pixel of an image and a second noise composed of noise for each pixel of the image correlated with the first noise are generated. The first noise is added to the left-eye image, and the second noise is added to the right-eye image.

  According to the present invention, it is possible to improve the quality of a 3D image when grain noise is added.

It is a figure which shows the method of adding grain noise. It is a figure which shows the image to which the grain noise was added by the method of FIG. It is a figure which shows the other method of adding a grain noise. It is a figure which shows the image to which the grain noise was added by the method of FIG. It is a block diagram which shows the structural example of the image processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the structural example of an image process part. It is a figure which shows the example of addition of random noise. It is a figure which shows the example of addition of a grain noise. It is another figure which shows the example of the image which added the grain noise. It is a figure explaining how a user looks. It is a top view explaining how a user looks. It is another top view explaining how a user looks. It is a flowchart explaining the display process of an image processing apparatus. It is a figure which shows the example of the setting of the parallax with respect to a grain noise. It is a figure explaining how a user looks. It is a top view explaining how a user looks. It is a flowchart explaining the other display processing of an image processing apparatus. It is a figure which shows the other structural example of an image process part. It is a figure which shows the example of L image and R image. It is a figure which shows the example of the grain noise for L images and R images. It is a figure which shows the example of L image and R image which added the grain noise. It is a figure explaining how a user looks. It is a top view explaining how a user looks. 12 is a flowchart illustrating still another display process of the image processing apparatus. It is a figure explaining how a user looks. It is a top view explaining how a user looks. It is a block diagram which shows the structural example of a computer.

[Configuration of image processing apparatus]
FIG. 5 is a block diagram illustrating a configuration example of the image processing apparatus 1 according to an embodiment of the present invention.

  The image processing apparatus 1 includes an acquisition unit 11, a decoder 12, an image processing unit 13, a display control unit 14, and a display 15.

  The acquisition unit 11 acquires 3D content recorded on a recording medium such as a Blu-ray (registered trademark) disc attached to the image processing apparatus 1 and 3D content transmitted via a broadcast wave or a network. The acquisition unit 11 outputs video data included in the acquired 3D content to the decoder 12. Video data of 3D content is composed of L image data and R image data, and is compressed by a method such as H.264 AVC (Advanced Video Coding) / MVC (Multi-view Video coding).

  The decoder 12 decodes the video data supplied from the acquisition unit 11 and generates an L image and an R image. The decoder 12 outputs the L image and the R image to the image processing unit 13.

  The image processing unit 13 generates a plurality of random noises having no correlation, and generates two random noises having a predetermined correlation based on the generated plurality of random noises. The image processing unit 13 adds one random noise to the L image as grain noise for the L image, and adds the other random noise to the R image as grain noise for the R image. The image processing unit 13 outputs the L image and the R image to which the grain noise is added to the display control unit 14.

  The display control unit 14 causes the display 15 to alternately display the L image and the R image supplied from the image processing unit 13.

  The image processing apparatus 1 having the above configuration is provided in, for example, an editing apparatus that edits 3D content, a television receiver that displays 3D content, and a recording apparatus that records 3D content.

  FIG. 6 is a diagram illustrating a configuration example of the image processing unit 13.

  The image processing unit 13 includes random noise generating units 21A to 21E, an adding unit 22, an adding unit 23, an adding unit 24, a grain noise adding unit 25, and a grain noise adding unit 26. The L image output from the decoder 12 is input to the grain noise adding unit 25, and the R image output from the decoder 12 is input to the grain noise adding unit 26.

  As indicated by the broken lines, the random noise generators 21A to 26E and the adders 22 to 24 realize the generation unit 31 as a generation unit that generates respective grain noises for the L image and the R image. Further, an adding unit 32 as an adding unit that adds grain noise for L image to each pixel of the L image and adds grain noise for R image to each pixel of the R image from the grain noise adding units 25 and 26. Realized.

  The random noise generators 21A to 21E generate random noises that are not correlated with each other. The random noise generated by the random noise generators 21A to 21E is a collection of randomly selected values corresponding to each pixel of the L / R image to which the grain noise is added. Each value represents a noise component for each pixel of the L / R image.

  By placing each randomly selected value as a pixel value at the same position as the pixel position of the corresponding L / R image, the random noise has the value of the noise component as the pixel value of each pixel, It is represented as a noise image having the same size as the L / R image.

  Here, the random noise generated by the random noise generating unit 21A will be described as random noise A, the random noise generated by the random noise generating unit 21B will be described as random noise B, and the random noise generated by the random noise generating unit 21C will be described as random noise C. . The random noise generated by the random noise generating unit 21D will be described as random noise D, and the random noise generated by the random noise generating unit 21E will be described as random noise E.

  The random noise A generated by the random noise generating unit 21A is supplied to the adding unit 23. The random noise B generated by the random noise generator 21B, the random noise C generated by the random noise generator 21C, and the random noise D generated by the random noise generator 21D are supplied to the adder 22. The random noise E generated by the random noise generator 21E is supplied to the adder 24.

  The adder 22 adds random noise B, random noise C, and random noise D. Random noise is added by adding the values of noise components at the same position (pixel values at the same position on the noise image). The adding unit 22 outputs random noise obtained by adding the random noise B, the random noise C, and the random noise D to the adding unit 23 and the adding unit 24.

  The adding unit 23 adds the random noise A and the random noise as a result of the addition by the adding unit 22, and outputs the random noise obtained by the addition to the grain noise adding unit 25 as the grain noise for the L image.

  The adding unit 24 adds the random noise E and the random noise as a result of the addition by the adding unit 22, and outputs the random noise obtained by the addition to the grain noise adding unit 26 as a grain noise for the R image.

  The grain noise for the L image and the grain noise for the R image each include a common component of the addition result of the random noise B, the random noise C, and the random noise D. Although the grain noise for the L image and the grain noise for the R image are not the same, they are noises correlated with each other.

  The grain noise adding unit 25 adds grain noise for L image supplied from the adding unit 23 to the L image supplied from the decoder 12. Adding the grain noise to the L image is performed by adding the value of the noise component at the same position included in the grain noise for the L image to the pixel value of each pixel of the L image. The grain noise adding unit 25 outputs the L image added with the L noise for the L image to the display control unit 14.

  The grain noise adding unit 26 adds grain noise for the R image supplied from the adding unit 24 to the R image supplied from the decoder 12. Adding the grain noise to the R image is performed by adding the value of the noise component at the same position included in the grain noise for the R image to the pixel value of each pixel of the R image. The grain noise adding unit 26 outputs the R image to which the grain noise for the R image is added to the display control unit 14.

  FIG. 7 is a diagram illustrating an example of random noise addition.

  In the example of FIG. 7, the random noise A is “1, 8, 2, 7, 3, 6, 4, 5,. Each number is a value representing a randomly selected noise component.

  Similarly, in the example of FIG. 7, the random noise B is “8,3,5,1,2,7,4,6,...” And the random noise C is “3,6,2,5,4”. , 1,8,7, ... ", random noise D is" 5,2,7,3,6,4,1,8, ... "and random noise E is" 4,7,1,6 , 5,8,7,2, ... ". Random noises A to E are uncorrelated noises in which the same numbers do not appear at the same positions in the arrangement order.

  In the adding unit 22, random noise B, random noise C, and random noise D are added. The random noise of the addition result is `` 16 (8 + 3 + 5), 11 (3 + 6 + 2), 14 (5 + 2 + 7), 9 (1 + 5 + 3), 12 (2 + 4 + 6), 12 (7 + 1 + 4), 13 (4 + 8 + 1), 21 (6 + 7 + 8),.

  In the adding unit 23, the random noise A and the random noise resulting from the addition by the adding unit 22 are added. The random noise of the addition result by the adding unit 23 is “17 (1 + 16), 19 (8 + 11), 16 (2 + 14), 16 (7 + 9), 15 (3 + 12), 18 (6 +12), 17 (4 + 13), 26 (5 + 21),. The random noise “17, 19, 16, 16, 15, 18, 17, 26,...” Obtained by the adding unit 23 is used as the grain noise for the L image.

  In the adding unit 24, the random noise E and the random noise resulting from the addition by the adding unit 22 are added. The random noise of the addition result by the adder 24 is “20 (4 + 16), 18 (7 + 11), 15 (1 + 14), 15 (6 + 9), 17 (5 + 12), 20 (8 +12), 20 (7 + 13), 23 (2 + 21),... The random noise “20, 18, 15, 15, 17, 20, 20, 23,...” Obtained by the adding unit 24 is used as the grain noise for the R image.

  In this way, the two grain noises for the L image and the R image having a predetermined correlation generate n random noises (n> 2) that are not correlated with each other, and any m (n−2) random noises among them are generated. It is generated by adding n-1 random noises so that the random noises of 1> m> 0) are included in common.

  In the example of FIG. 7, n = 5 and m = 3. The two grain noises for the L image and the R image are the addition results of four random noises including the addition result of three random noises in common. Since the addition result of the three random noises is included in the addition result of the four random noises, the correlation rate between the grain noise for the L image and the grain noise for the R image is 3/4 = 75%. Become. When n> 2 and n-1> m> 0 and the values of n and m are arbitrarily selected, the correlation rate between the grain noise for the L image and the grain noise for the R image is expressed by m / n-1. .

  FIG. 8 is a diagram illustrating an example of adding grain noise.

  The waveform in FIG. 8A shows the waveform of an L image of a certain frame. The horizontal axis of each graph in FIG. 8 represents the pixel position in raster order, and the vertical axis represents the pixel value.

  When the size of the L image is 1920 × 1080 pixels, the value of the position P0 is the pixel value of one pixel at the position represented by (0, 0) at the upper left corner of the L image, and the value of the position P1 is the L image. Is the pixel value of one pixel at the position represented by (1919,0) at the upper right end of. The value of the position P2 is the pixel value of one pixel at the position represented by (1919, 1079) at the lower right corner of the L image. The pixel value represents RGB with 8 bits each.

  The waveform in FIG. 8B shows the waveform of the grain noise for the L image generated by the adding unit 23. As shown in FIG. 8B, the grain noise is composed of random values represented by a predetermined bit width.

  The value of the grain noise position P0 for the L image is the value of the noise component added to one pixel at the position represented by (0, 0) at the upper left corner of the L image, and the value of the position P1 is L This is the value of the noise component added to one pixel at the position represented by (1919,0) in the upper right corner of the image. The value of the position P2 is the value of the noise component added to one pixel at the position represented by (1919,1079) at the lower right corner of the L image.

  The waveform of FIG. 8C shows the waveform of the L image to which the grain noise is added. By adding the value of the noise component at the same position of the grain noise shown in FIG. 8B to the pixel value of each pixel of the L image shown in FIG. 8A, L having the waveform shown in FIG. An image is obtained.

  In the grain noise adding unit 26, the same processing as that performed on the L image is performed on the R image.

  FIG. 9 is a diagram illustrating an example of an L image and an R image to which grain noise is added.

  Random noise as a result of addition by the adding unit 23 is added as grain noise for the L image to the image L11 which is an L image, and an image L11 'is generated. Further, random noise as a result of addition by the adding unit 24 is added as grain noise for the R image to the image R11 which is an R image, and an image R11 'is generated. There is a shift corresponding to parallax between the position of the object in the image L11 and the position of the object in the image R11. In the example of FIG. 9, a person is shown in the image L11 and the image R11.

  In the images L11 'and R11', the pattern added to the background of the person indicates grain noise. The person area is not patterned, but grain noise is added to the entire image including the person area. In FIG. 9, the grain noise added to the image L11 and the grain noise added to the image R11 are shown in the same pattern, but in reality, they are different noises that have a correlation. The same applies to the other drawings.

  The image L11 'to which the grain noise for the L image is added and the image R11' to which the grain noise for the R image is added are alternately displayed as indicated by the tip of arrow # 11.

  FIG. 10 is a diagram for explaining how the user looks.

  Since there is a predetermined correlation between the grain noise for the L image and the grain noise for the R image, noise having slightly different values is added to the same position in the image L11 and the image R11. A slight difference between the noise added to the image L11 and the noise added to the image R11 acts as parallax.

  In the case of the example in FIG. 7, “17, 19, 16, 16, 15, 18, 17, 26,...” That is the grain noise value for the L image, and the grain noise value for the R image. "3,1,1,1,2,2,3, ..." which is the absolute value of the difference from "20,18,15,15,17,20,20,23, ..." Will act as parallax.

  A slight difference in noise acts as parallax, so that the grain noise that is localized on the display surface D appears three-dimensionally to a user wearing 3D glasses. The center plane G is the center plane of grain noise, and coincides with the display plane D in this example. Moreover, since the parallax is set, the user can see the person three-dimensionally.

  In this way, by generating two random noises having a predetermined correlation and adding them to the L image and the R image respectively, it is possible to express the grain noise with a stereoscopic effect on the display surface D by localization, This makes it possible to enhance the fineness and gradation of the image.

  FIG. 11 is a top view for explaining how the user looks.

  In the L image that reaches the user's left eye, the display position of the person is shifted to the right from the reference position (position to be displayed), and in the R image that reaches the user's right eye, the display position of the person is left from the reference position. By shifting in the direction, the person appears to jump out of the display surface D.

  Grain noise appears to be distributed in a relatively narrow space before and after the display surface D. This is because there is little variation in noise added to the L image and the R image because there is a correlation between the grain noise for the L image and the grain noise for the R image. The dots shown above and below the display surface D represent grain noise that appears three-dimensionally.

  FIG. 12 is another top view for explaining how the user looks.

  In the L image, the person's display position is shifted to the left from the reference position, and in the R image, the person's display position is shifted to the right from the reference position, so that the person appears to be behind the display surface D. It will be. Even when a person appears to be in the back of the display surface D, when the grain noise generated as described above is added to the L image and the R image, the grain noise is relatively less before and after the display surface D. It seems to be distributed in a narrow space.

  As described above, the grain noise is localized on the display surface D and displayed in a three-dimensional manner, thereby enhancing the sense of fineness and the gradation and improving the image quality of the 3D image.

[Operation of Image Processing Device]
Here, the processing of the image processing apparatus 1 for displaying a 3D image will be described with reference to the flowchart of FIG.

  In step S1, the acquisition unit 11 acquires 3D content by reading from the recording medium.

  In step S2, the decoder 12 decodes video data included in the 3D content, and generates an L image and an R image.

  In step S3, the random noise generation units 21A to 21E of the image processing unit 13 generate a plurality of random noises having no correlation.

  In step S4, the adding unit 22 of the image processing unit 13 adds the random noises B, C, and D. Further, the adding unit 23 adds the random noise A and the addition result of the random noises B, C, and D to generate grain noise for the L image. The adder 24 adds the random noise E and the addition result of the random noises B, C, and D to generate grain noise for the R image.

  In step S5, the grain noise adding unit 25 adds grain noise for the L image to the L image. Further, the grain noise adding unit 26 adds grain noise for the R image to the R image.

  In step S6, the display control unit 14 causes the display 15 to alternately display the L image and the R image to which the grain noise is added, and ends the process.

[Modification 1]
When the two random noises generated as described above are directly added as the grain noise for the L image and the R image, the grain noise is localized on the display surface D as described with reference to FIG. Will be seen.

  For example, in 3D content and scenes with a lot of depiction in the depth direction, it is more advantageous to increase the grain noise deeper than the display surface D in order to increase the sense of detail and gradation, or to reduce the sense of incongruity. To do.

  Parallax may be set for the grain noise for the L image and the grain noise for the R image, and the grain noise may be localized behind the display surface D.

  FIG. 14 is a diagram illustrating an example of setting parallax for grain noise.

  An image L11 ′ in FIG. 14 is an L image to which grain noise for L image shifted to the left with respect to the position of the L image is added. The range of the width W1 is the range of the L image (image L11). In the example described with reference to FIG. 9, the grain noise for the L image is added to the same range as the range of the width W1, whereas the grain noise for the L image is entirely added in the example of FIG. In particular, the left side is shifted by a width W2.

  In other words, in this case, the grain noise adding unit 25 sets the parallax to the grain noise for the L image by shifting the entire noise image representing the grain noise for the L image to the left by the pixel corresponding to the width W2. To do.

  Further, the grain noise adding unit 25 is not the value of the noise component at the same position of the grain noise for the L image but the pixel value of each pixel of the L image by the pixel corresponding to the width W2 on the right side. By adding a noise component at a shifted position, grain noise is added to the L image. For example, the grain noise adding unit 25, for the pixel value of the pixel at the position represented by (0, 0) at the upper left end of the L image, out of the pixels of the noise image representing the grain noise for the L image, The pixel value at the position represented by (α, 0) is added instead of the pixel value (noise component) at the position represented by the same (0,0). α is a pixel corresponding to the width W2.

  An image R11 'in FIG. 14 is an R image to which grain noise for the R image shifted to the right with respect to the position of the R image is added. The range of the width W1 is the range of the R image (image R11). In the example described with reference to FIG. 9, the grain noise for the R image is added to the same range as the range of the width W1, whereas in the example of FIG. Thus, the right side is shifted by a width W2.

  In this case, the grain noise adding unit 26 sets the parallax to the grain noise for the R image by shifting the entire noise image representing the grain noise for the R image to the right by the pixel corresponding to the width W2.

  Further, the grain noise adding unit 26 is not the value of the noise component at the same position of the grain noise for the R image with respect to the pixel value of each pixel of the R image, but on the left side by the pixel corresponding to the width W2. By adding a noise component at a shifted position, grain noise is added to the R image.

  The images L11 'and R11' to which the grain noise with parallax is added are alternately displayed as indicated by the tip of arrow # 21.

  FIG. 15 is a diagram for explaining how the user looks.

  Since there is a parallax in the grain noise, the three-dimensional grain noise appears to be localized behind the display surface D to the user wearing 3D glasses. The range d represents the range on the display surface D of the grain noise that the user can see.

  As described with reference to FIG. 12, in the L image, the display position is shifted to the left from the reference position, and in the R image, the display position is shifted to the right from the reference position. It is possible to make it appear to be in the back. When parallax is set in such a manner, the grain noise also appears to be behind the display surface D.

  FIG. 16 is a top view for explaining how the user looks.

  The position of the center plane G of grain noise is a position behind the display plane D. Grain noise seems to be distributed in a relatively narrow space before and after the central plane G.

  Here, with reference to the flowchart of FIG. 17, processing of the image processing apparatus 1 that displays 3D images by adding grain noise with parallax to the L and R images will be described.

  The processes in steps S11 to S14 in FIG. 17 are the same as the processes in steps S1 to S4 in FIG. That is, in step S11, the acquisition unit 11 acquires 3D content.

  In step S12, the decoder 12 decodes the video data.

  In step S13, the image processing unit 13 generates a plurality of random noises having no correlation.

  In step S <b> 14, the image processing unit 13 generates grain noise for L image and grain noise for R image based on a plurality of random noises having no correlation.

  In step S15, the grain noise adding unit 25 of the image processing unit 13 sets parallax to the grain noise for the L image. Further, the grain noise adding unit 26 sets a parallax for the grain noise for the R image.

  In step S <b> 16, the grain noise adding unit 25 adds grain noise for L image with parallax set to the L image. Further, the grain noise adding unit 26 adds grain noise for the R image in which the parallax is set to the R image.

  In step S <b> 17, the display control unit 14 causes the display 15 to alternately display the L image and the R image to which the grain noise set with the parallax is added, and ends the processing.

  The parallax is not set for both of the L noise for the L image and the grain noise for the R image, but the parallax is set for one of the grain noise for the L image and the grain noise for the R image. You may be made to do.

  Further, the parallax set for the grain noise may dynamically change according to the change in the parallax between the L image and the R image so that the parallax is considered to be optimal for obtaining the effect of the grain noise. . In this case, detection of parallax between the L image and the R image is repeatedly performed. The parallax between the L image and the R image is represented by, for example, an average of parallax between objects of the L image and the R image. The grain noise adding unit 25 and the grain noise adding unit 26 change the parallax according to the detected parallax between the L image and the R image and set the grain noise.

[Modification 2]
In the above, the same amount of parallax is set for the entire grain noise, but the object of the L image and the R image is detected, and the same amount of parallax as that set for the object is added to the object. You may make it set to noise.

  FIG. 18 is a diagram illustrating another configuration example of the image processing unit 13.

  Of the configurations shown in FIG. 18, the same configurations as those shown in FIG. The overlapping description will be omitted as appropriate.

  The configuration of the image processing unit 13 illustrated in FIG. 18 is the same as the configuration illustrated in FIG. 6 except that an object detection unit 41 is additionally provided. The L image and R image output from the decoder 12 are also input to the object detection unit 41.

  The object detection unit 41 analyzes the L image, detects an edge, and detects an object included in the L image. Similarly, the object detection unit 41 analyzes the R image and detects an object included in the R image.

  The object detection unit 41 detects, as parallax, a shift amount between the position of the object detected from the L image in the L image and the position of the object detected from the R image in the R image. Among the objects included in the R image, which object is the same object as the object included in the L image is specified by performing block matching, for example. About the detection of parallax, it is disclosed by Unexamined-Japanese-Patent No. 7-282259, for example.

  The object detection unit 41 outputs the object position (range) and parallax information detected from the L image to the grain noise addition unit 25, and the object position and parallax information detected from the R image to the grain noise addition unit 26. Output.

  The grain noise adding unit 25 specifies the position of the object included in the L image supplied from the decoder 12 based on the information supplied from the object detecting unit 41, and specifies the parallax set for the object. .

  In addition, the grain noise adding unit 25 is set to the object instead of the value of the noise component at the same position of the grain noise for the L image with respect to the pixel value of each pixel of the L image constituting the object region. A noise component at a position shifted by the amount of parallax is added. The grain noise for the L image is supplied from the adding unit 23 to the grain noise adding unit 25.

  Based on the information supplied from the object detection unit 41, the grain noise addition unit 26 specifies the position of the object included in the R image supplied from the decoder 12, and specifies the parallax set for the object. .

  Further, the grain noise adding unit 26 is set to the object, not the value of the noise component at the same position of the grain noise for the R image, with respect to the pixel value of each pixel of the R image constituting the object region. A noise component at a position shifted by the amount of parallax is added. The grain noise for the R image is supplied from the adding unit 24 to the grain noise adding unit 26.

  The setting of parallax for grain noise performed in the grain noise adding unit 25 and the grain noise adding unit 26 will be described.

  FIG. 19 is a diagram illustrating an example of an L image and an R image.

  The size of the image L21, which is an L image, and the image R21, which is an R image, is 1920 × 1080 pixels, respectively. A case will be described in which parallax is set for the grain noise added to the pixels constituting the region of the human face, which is surrounded by a circle C, among the pixels in the x-th row of the image L21 and the image R21. Here, the parallax is set only for the grain noise for the R image.

  Of the pixels in the x-th row of the image L21, the range of the pixels constituting the human face region is from the nth pixel to the n + β pixel. In addition, among the pixels in the x-th row of the image R21, the range of the pixels constituting the human face region is from the m-th pixel to the m + α pixel. The parallax set between the face of the person in the image L21 and the face of the person in the image R21 is represented by, for example, nm.

  FIG. 20 is a diagram illustrating an example of a noise image representing grain noise.

  The image L31 is an image representing grain noise for the L image, which is an addition result by the adding unit 23. The image R31 is an image representing grain noise for the R image, which is an addition result by the adding unit 24. The sizes of the image L31 and the image R31 are also 1920 × 1080 pixels.

As shown as the pixel value of the image L31, the noise component in the first row is “a ₀ , a ₁ , a ₂ ,..., A ₁₉₁₉ ”, and the noise component in the x row is “a _x , a _{x +1} , ..., a _m , ..., a _{m + α} , ..., a _n , ..., a _{n + β} , ..., a ₁₉₁₉ ".

As shown as the pixel value of the image R31, the noise component in the first row is “b ₀ , b ₁ , b ₂ ,..., B ₁₉₁₉ ”, and the noise component in the x row is “b _x , b _{x. +1, ···, b m, ···} , b m + α, ···, b n, ···, b n + β, ···, it is a b ₁₉₁₉ ".

  FIG. 21 is a diagram illustrating an example of an L image and an R image to which grain noise is added.

As illustrated in an image L21 ′ in FIG. 21, the grain noise adding unit 25 includes each of the pixels from the nth pixel to the n + βth pixel constituting the human face area among the pixels in the xth row of the image L21. “A _n ,..., A _{n + β} ”, which are noise components at the same position in the image L31 (FIG. 20), are added to these pixels. In this example, since no parallax is set for the grain noise for the L image, a noise component at the same position of the grain noise for the L image is added to each pixel of the L image.

Further, as shown in an image R21 ′ in FIG. 21, the grain noise adding unit 26 from the m-th pixel to the m + α-th pixel constituting the face area of the person among the pixels in the x-th row of the image R21. Are not the noise components “b _m ,..., B _{m + α} ” which are noise components at the same position in the image R31 (FIG. 20), but are shifted by the parallax nm set on the face. “B _n ,..., B _{n + β} ” as noise components are added. That is, a noise component in which the same amount of parallax as the parallax set for the object is added to the pixels constituting the object region.

  The image L21 'and the image R21' to which the grain noise has been added in this way are alternately displayed as indicated by the tip of the arrow # 32.

  In the grain noise adding unit 25 and the grain noise adding unit 26, the same parallax as the parallax set for the object is set in the same manner for each noise added to the pixels constituting all object regions. It is added to the pixels constituting the region.

  FIG. 22 is a diagram illustrating how the user looks.

The noise component “a _n ,..., A _{n + β} ” added to the pixels from the nth pixel of the xth row of the image L21 to the n + βth pixel, and the xth row of the image R21 The noise components “b _n ,..., b _{n + β} ” added to the pixels from the m-th pixel to the m + α-th pixel are noise components at the same position in the image L31 and the image R31. , There is a correlation with each other. Therefore, these noise components make the grain noise look three-dimensional.

Further, correlated to each other, _{"a n, ···, a n +} β " position of the noise component and the _{"b n, ···, b n +} β " to the position of the noise component of it is added There is the same shift as the parallax set on the face of the person displayed by the pixels. Therefore, grain noise can be seen by these noise components so as to be localized at the same position as the position of the face of the person who appears stereoscopically. In FIG. 22, the hatched area of the person's face indicates that grain noise can be seen at the same position as the face position.

  Since no parallax is set for the noise component added to the pixels other than the pixels constituting the object area, grain noise other than the grain noise localized at the position of the object appears to be localized on the display surface D. Become.

  FIG. 23 is a top view for explaining how the user sees.

  A plane G1 shown on an ellipse representing a person is a central plane of grain noise added to the pixels constituting the face area of the person. Grain noise seems to be distributed in a relatively narrow space before and after the central plane G1. The other parts of FIG. 23 are the same as those described with reference to FIG.

  Although the case where the parallax is set only for the grain noise for the R image has been described, the parallax may be set only for the grain noise for the L image. Further, the parallax may be set for the noise components of the grain noise for both the L image and the R image.

  Here, referring to the flowchart of FIG. 24, parallax is set for the grain noise added to the pixels constituting the object region, and the 3D image is displayed by adding the grain noise with parallax to the L image and the R image. Processing of the image processing apparatus 1 will be described.

  The process in FIG. 24 is basically the same as the process in FIG. 13 except that a process for detecting an object and a process for setting a parallax for grain noise are added. That is, in step S21, the acquisition unit 11 acquires 3D content.

  In step S22, the decoder 12 decodes the video data.

  In step S23, the object detection unit 41 detects an object by analyzing the L image and the R image. The object detection unit 41 outputs the object position and parallax information detected from the L image to the grain noise addition unit 25, and outputs the object position and parallax information detected from the R image to the grain noise addition unit 26.

  In step S24, the image processing unit 13 generates a plurality of random noises having no correlation.

  In step S25, the image processing unit 13 generates grain noise for L image and grain noise for R image based on a plurality of random noises having no correlation.

  In step S <b> 26, the grain noise adding unit 25 of the image processing unit 13 applies an object to the noise component added to the pixels of the L image that constitutes the object region, out of the noise components of the grain noise for the L image. Set the same parallax as the set parallax. The grain noise adding unit 25 adds a noise component for which parallax is set to the pixels of the L image constituting the object region.

  Further, the grain noise adding unit 26 has the same parallax as the parallax set for the object with respect to the noise component added to the pixel of the R image constituting the region of the object among the noise components of the grain noise for the R image. Set. The grain noise adding unit 26 adds a noise component for which parallax has been set to pixels of an R image constituting the object region.

  In step S27, the display control unit 14 causes the display 15 to alternately display the L image and the R image to which the grain noise to which parallax is set is added, and ends the processing.

  Through the above processing, it is possible to display the grain noise so that it is localized at the same position as the position of the object.

  The parallax is set only for the grain noise added to the pixels constituting the object region among the pixels of the L image and the R image. However, the parallax may be set for other grain noises. .

  For example, the parallax can be set as described with reference to FIG. 14 for the grain noise added to the pixels other than the pixels constituting the object region. For the grain noise added to the pixels constituting the object region, the parallax is set as described with reference to FIGS.

  FIG. 25 is a diagram for explaining how the user looks when the parallax is set as described with reference to FIG. 14 for the grain noise added to the pixels other than the pixels constituting the object region. is there.

  In this case, as in the case described with reference to FIG. 22, the user wearing 3D glasses will see grain noise localized at the position of the person's face. In addition, since the parallax is also set for the grain noise added to the pixels constituting the background, which are pixels other than the pixels constituting the object region, the user can see from the display surface D in the background range. Grain noise appears to be localized in the back.

  FIG. 26 is a top view for explaining how the user looks.

  A plane G1 shown on an ellipse representing a person is a central plane of grain noise added to the pixels constituting the face area of the person. Since the parallax is set, the position of the center plane G of the grain noise is located behind the display plane D in the background range.

[Computer configuration example]
The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

  FIG. 27 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

  A CPU (Central Processing Unit) 51, a ROM (Read Only Memory) 52, and a RAM (Random Access Memory) 53 are connected to each other via a bus 54.

  An input / output interface 55 is further connected to the bus 54. Connected to the input / output interface 55 are an input unit 56 such as a keyboard and a mouse, and an output unit 57 such as a display and a speaker. The input / output interface 55 is connected to a storage unit 58 made up of a hard disk, a non-volatile memory, etc., a communication unit 59 made up of a network interface, etc., and a drive 60 that drives the removable media 61.

  In the computer configured as described above, for example, the CPU 51 loads the program stored in the storage unit 58 to the RAM 53 via the input / output interface 55 and the bus 54 and executes it, thereby executing the above-described series of processing. Is done.

  The program executed by the CPU 51 is recorded in, for example, the removable medium 61 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and is installed in the storage unit 58.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 13 Image processing part, 21A thru | or 21E Random noise generation part, 22 thru | or 24 addition part, 25,26 Random noise addition part, 41 Object detection part

Claims (7)

Generating means for generating a first noise composed of noise for each pixel of the image and a second noise composed of noise for each pixel of the image correlated with the first noise;
An image processing apparatus comprising: an adding unit that adds the first noise to the left-eye image and adds the second noise to the right-eye image. The generating means generates n (n> 2) uncorrelated noise, and is a sum of arbitrary m (n-1>m> 0) noises out of the n uncorrelated noises. The image processing apparatus according to claim 1, wherein the first noise and the second noise having a correlation of m / (n−1) that include the components represented in common are generated. The image processing apparatus according to claim 1, further comprising display control means for alternately displaying the left-eye image to which the first noise is added and the right-eye image to which the second noise is added. The adding means sets a parallax for the first noise and the second noise, and the first noise and the second noise having parallax with each other are respectively applied to the left-eye image and the right-eye image. The image processing apparatus according to claim 1 to be added. A detecting means for detecting an object included in the left-eye image and the right-eye image;
The adding means includes noise added to each pixel constituting the object area detected from the left-eye image and noise added to each pixel constituting the object area detected from the right-eye image. The image according to claim 1, wherein at least one of them sets a parallax of the same amount as the parallax set for the object, and adds noise with the parallax set to each pixel constituting the region of the object. Processing equipment. Generating a first noise consisting of noise for each pixel of the image and a second noise consisting of noise for each pixel of the image correlated with the first noise;
An image processing method comprising: adding the first noise to a left-eye image and adding the second noise to a right-eye image. Generating a first noise consisting of noise for each pixel of the image and a second noise consisting of noise for each pixel of the image correlated with the first noise;
A program for causing a computer to execute a process including a step of adding the first noise to a left-eye image and adding the second noise to a right-eye image.

Images